Well, surprisingly well in fact. I say ‘surprisingly’ not because it was a casual throw-of-the-dice affair - I did a lot of preparation and testing - but because I’m old enough to know that nothing ever goes completely to plan, and we didn’t have any (cost effective) way to test the full server load ahead of turning it on. So as the DNS update slowly propagated across the globe last Saturday, I was sitting nervously at my desk watching the server load with my fingers crossed.
There was one early blip beyond the Tikiwiki shortlink compatibility issue we already knew about - occasionally there was a transient 500 error on the front page of the site (or occasionally in one of the sub-pages), which I eventually traced to the WordPress cacheing plugin we used. I still don’t know why it happened, or why it would randomly appear and disappear (it didn’t sync up entirely with the cache expiry, but maybe it was a combination of time-related factors and which nginx/php-fm process managed the request at the time), but disabling the cacheing eliminated it and did not seem to significantly affect the server load. More investigation on that one at a later date, but for now the site managed just fine without it.
The best thing from my perspective has to be the memory consumption behaviour of Nginx & PHP-FPM. The VPS we’re using actually only has 768MB of RAM - not very large for a fairly busy site with almost entirely dynamic content (because the community forums / wiki are the most popular parts). But I knew that it was easy to add more RAM if I needed it later on Linode, so I figured I’d give it a try and see whether the hype about Nginx was true.
As the traffic started to build, the memory dropped quite significantly, quickly sitting at less than 100MB free. But here’s the awesome thing - despite bouncing around below that level quite a lot, sometimes into single figures and therefore making me really anxious, it never once got into the kind of swap-space thrashing behaviour that Apache so often did (and which was prone to killing the site performance under load spikes). I’d done my sums so I was fairly confident I’d budgeted correctly for the number of Nginx processes and the PHP-FPM pool, but I don’t think I could have got it that accurate - I do wonder if either of them or both are smart enough to figure out how to behave in low memory situations on their own. Either way, the result is that I’ve never seen it eat any swap space yet, despite grazing the horizon frequently. The result is a very, very efficient system. Result!
CPU wise, there’s little to be concerned about too. Here’s the graph for a typical 24 hours:
To be fair, CPU was never a problem with Apache either, until it got into swapping problems.
From a user perspective, things have been great. We’ve had lots of comments saying the site feels snappier than it did before, despite that fact that by the raw numbers, it’s running on a less capable machine. It doesn’t get much better than that.
So, my conclusion here is that Nginx, PHP-FPM and APC make an awesome combination for running a busy site on a cost-efficient configuration. It also illustrates to me how far things have come - when we tried a VPS in 2005 with lighthttpd (still on Linode even then), the site was destroyed in no time at all, and certainly when running Apache even our fully dedicated server with extensive performance tweaks was not capable of handling unexpected spikes particularly well.
Most of the docs & online comments say that Nginx is really designed for serving static content or being a reverse proxy, so you should be careful of what you expect from it for other purposes, but coupled with PHP-FPM and APC my tests show it can definitely be a great server for dynamic content too.