Anyone who has used digg for any amount of time has certainly come across a dead link. Invariably you click on the comment link hoping someone has posted a mirror of the content. You head over to duggmirror or coral cache to view the content. During the peak time right after an article hits the digg homepage, a given site might receive as many as 100 page loads per minute. If this page is hosted on WordPress, or other MySQL based blog platform, combined with shared hosting, it’s a recipe for a crash. Your site will go down in flames and you won’t have a chance of recovering until the load goes back to a lower level.
Recently I had an article hit the front page of Digg. I was lucky because I had already done a number of things to optimize my server so that I survived. My page never went down, and was very responsive for the entire duration of getting dugg. My site is hosted on WordPress on a server that I own and manage. The server hosts about 15 different domains, none of which are exceptionally high traffic sites. The server isn’t anything special either. It’s a single processor P4 3.0GHz with 512MB of memory running CentOS 4.4, Apache 2.0, PHP5, MySQL4. I do have webalizer running on my Apache logs and I have the output of those posted at http://vallery.net/stats/ if you would like to see what the Digg effect can do.
I never exceeded more than a 0.15 load average during the Digg. Load average is a computation of how maxed out a Unix based system is. You multiply the number of processors by 1, and this indicates the maximum load average that you can reasonably sustain. If the load average exceeds this number, than your system is over utilized. In my case, my max server load is 1 given that this is a single processor system. This means I was only using 15% of my system resources during the maximum load that was generated from the Digg.
In order to optimize my system I have done the following things:
1) I’ve installed the WordPress 2.0 plugin wp-cache. This is something that every WordPress blog should have installed. Especially on a shared hosting environment, it will dramatically increase your ability to handle high traffic. The plugin generates the HTML for a given page and then saves it in a cache file. When someone accesses your page, instead of fetching the content from the database it uses the already generated HTML in the cache to send to the browser. This eliminates a number of fetches to the DB and dramatically speeds up page load times.
2) Optimize Apache/MySQL to handle the expected number of database queries. The most common error you receive when viewing a WordPress page that has been owned by the Digg effect is a database connection failed, or timeout. When connecting to a MySQL database you can either have a persistent connection, or generate a new connection for each request. There are a number of schools of thought as to which method is better, but generally speaking using persistent connections utilize more memory, so for me using a new connection for each request makes more sense. A lot of the PHP blog templates including WordPress use these non-persistent connections as well. The rub comes in the maximum number of said connections. MySQL can be configured in your my.cnf file to say the maximum number of connections that can be created to the database. If each instance of Apache (which in turn represents concurrent vistors to your page) has it’s own connection, then you can quickly exceed this maximum number of connections. Apache similarly has a configuration option that dictates the maximum number of threads that can be concurrently running. This number is specified in the httpd.conf file. If the Apache number is larger than the MySQL number (which is the case in most default configurations) when you have more Apache threads running than MySQL is capable of handling, you get our nice “maximum number of connections” error message.
In /etc/my.cnf you want to set the variable “max_connections”. I recommend something reasonably high like 250.
In your httpd.conf file for Apache you want to set the variable “MaxClients”. This should typically be the same number that you selected for MySQL.
3) Set reasonable Apache timeouts. This means that no individual thread/connection can monopolize the system bringing all the other queries to its knees. This protects you from rogue “edge” cases.
In your httpd.conf file set “Timeout” to a low number like 30 (measured in seconds) and “KeepAliveTimeout” to something like 3.
4) If your site makes it on to Digg, use real-time monitoring tools to measure your server’s health. There are a number of command line tools available to help in this regard. The first I would recommend is the tried and true “top” command. This will display the processes running on your machine along with their associated memory and CPU usage. Keep an eye and make sure things aren’t getting out of hand. The second tool that I use is called “Tcptrack“. Tcptrack will need to be installed on your machine, but once it is it will give you a real time view of your incoming connections and bandwidth usage.
Happy Digging!
Hi Vallery,
I would suggest that you install Akismet for your blog, because as I can see, most (if not all) of the comments on this article are comment-spam. I was thinking that they are real people, but then I realised that I am wrong. Akismet has caught ~900 of them so far and it is doing a great job at catching terrible comment-spam
Regards
I really love to try it out. This is something new
Молодец конечно! Целиком и полностью поддерживаю!
ceiling fans are great for cooling a warm room with large area. i prefer them over multiple desk fans-’”
I’m have slowness issue with my Toronto Wedding Photographers Blog and bumped into this site in an effort to increase response times. Thanks for the tips!
7 blade ceiling fans have much better performance compared to those three bladed fans ~,.
:.: I am very thankful to this topic because it really gives useful information ‘.: