Speeding up Drupal Forums

The Drupal forum.module has become, well, somewhat infamous for less than awesome scalability. Recently I had a chance to see this firsthand, and track down a solution for managing the long page load times for a client who has a highly trafficked forum. This was not a case of a site that was un-tuned - actually this particular site had a lot of good work and performance enhancements already done to it, including block caching and even some modifications to the forum module that were allowing to work better than it would have without them. But still 5-6 second page load times on /forum persisted.

As this was my first time working on the site, I began by reviewing all of the main configuration files for Apache, MySQL, and PHP, since they are the foundation for everything else. After making some adjustment there, I headed over to the Drupal admin interface and check /admin/setting/performance/ and made sure all was happy there as well. Finally, I went to the block admin page and double-checked all of the blockcache settings, which as it turns out were set a bit to aggressively, resulting in slow form submission times (every time anyone submitted anything a gazillion blocks were being re-cached whether they need to be or not).

With the foundation of the site now looking good and everything, except the forum pages flying along (and the tracker, but that's a story for a later article) - it was just down to forum.module.

Here are the steps that led to cutting the page load time in half from what they were:

1. Disable the forum.module, rename it to something like "forum.module.orig", make a copy and then rename the copy "forum.module".

2. Download the advcache module and apply the forum patch includes to your new copy of the forum.module.

3. If want the cleanest solution and are comfortable with coding/debugging at all, instead of just copying the forum module and working on it directly (and thus having a hacked Drupal core file around all the time) - name the copied file something completely different than forum.module and edit all of the hook/function calls with in it with new name and place it where you keep the rest of your contributed modules.

4. (Note: this step is an option only if all your forums are public)
Open the module and remove all references to db_rewrite_sql. This will keep Drupal from doing a lot of expensive and uneccesary queries in order to check access rights. (thanks Khalid)

28 February, 2008

Drupal 6: Benchmarking and Block Cache Performance Revisited

As a follow up to an earlier article I posted about Drupal 6 performance, and please bear with my learning curve for a moment, I figured out by 'accident', and a lot of investigation, that it matters very much the order one uses when they 'generate content' with the devel module for benchmarking purposes. My previous tests were done incorrectly - I inadvertently created a bunch of nodes that weren't assigned to any terms or users and vice versa. The result of correcting this error means that a no-cache-enabled-baseline takes much longer to complete than when I had things setup incorrectly.

...happily, the point of this article isn't that I'm a total goof.

No, the good news out of this ordeal is that now when block-cache-disabled performance is compared to block-cache-enabled performance the results are MUCH more substantial than previously noted (and thus Drupal 6 is going to be that much faster than it's predecessor Drupal 5 for authenticated users):

2489.69 ms (request time for auth user, no-caching of any kind)
-878.09 ms (request time for auth user, block-caching on)
1,611.6 (difference) / 2489.69 =
64.73% improvement w/ block cache on

With the block caching on, the mean processing time is 876 ms with a sd of 91.9 ms while the base install results in 2481 ms mean processing time and sd of 91.9. Even at the upper end of the standard deviation, the block-cached processing time is 967.9 ms, which is far below the low end of the standard deviation (2080.1 ms) for the non-block-cached test. Looks like a clear improvement - 64.7 percent improvment using just the means.

The benchmarks are posted here so that everyone can do their own math. If you'd like to check the validity of my installation/numbers - feel free to download a tarball which includes all the files and a db dump. Username/pass for user 1 = superadmin


Benchmarks using 10,000 nodes, 5000 comments, 15 categories, 250 terms, 2000 users and with the following blocks enabled:

Recent comments 
User login
Active forum topics
New forum topics
Who's online


13 August, 2007
Subscribe to RSS - tuning