Oh dear, this is really not good at all. Yesterday morning Sourceforge had a hardware failure on their developer CVS server which has effectively ground all projects hosted by it to a halt. They are currently reporting that whilst they are working on it, they have no firm fix timescale and that it will be at least 48 hours before it is back, at a minimum. This means OGRE 1.2.0 RC2 cannot happen this weekend.
I’m actually very surprised that Sourceforge have a single server hosting developer CVS anyway. I assumed with the number of projects they host, they would spread this load across more than one machine. Perhaps they only do this for the anonymous read-only CVS services (which are still up) since that probably generates more traffic.
Now, many people are going to bash them for this. It’s partly deserved, since the fact that something could take out their most important service (the crucial developer source code hosting) for this long is a poor show indeed. However, some restraint is in order, since it’s not like the service has actually cost us anything. I do tend to defend Sourceforge a lot when people bash them, since I really appreciate the service they provide. I mean, without their help it would have been impossible to launch OGRE as a project, and their downloads service buffers us against the 7-800Gb/month traffic we’d otherwise have to shoulder (and goodness knows how much the anonymous CVS service absorbs). So we’re getting an awful lot for free. We moved our web hosting onto a dedicated server a year ago because it was clear the our traffic demands were outstripping the shared service Sourceforge provide, but everything else is managed by them, and I’ve been grateful for that.
That said, it’s clear that Sourceforge is suffering a deterioration. At one time, support appears to have been almost 24×7 - perhaps not quite, but I definitely got support on the weekend in the early days. Now, their engineers tend to only work weekdays and regular hours. They also have either cut down on engineers, or have lost some, to the extent that they warned that support may be degraded until 2 April because of lack of staff, and they’re advertising right now. I hear anecdotally that a year or two ago, funding and support to the free Sourceforge service was significantly cut by the parent company. Over the past 6 months there have been several outages which have taken 12 hours or more to resolve. For any serious project, this is not good.
So, much as I have loved Sourceforge despite the occasional warts, I think we will need to start making plans to move our source code repository somewhere else. This is not going to be cheap - our current web server has no spare capacity and no official backup service (just scripted ones of the database and ad-hoc content backups) - therefore it probably can’t be funded right now from our existing donations, which are only just enough to fund our existing server. I hope that the license extensions in Eihort will bring in some additional funds that, among other things, we can use to run a larger server with a dedicated source code repository, with a daily automated backup service. I’m sure we’ll still use Sourceforge for download mirroring (no point burning bandwidth if we don’t have to), but the source code system is too crucial to be out of our control if something like this can happen.
Yes, in theory we could switch to another open source provider (like Berlios), but there’s no guarantee this kind of thing won’t happen there too. And thanks for the tip about the OSDL funding option Chris, although I think due to the fact that we’d like to support consoles too, we’re probably not quite as free-software purist as the projects they’d be looking for. Besides, I’d like to control my own destiny rather than beg for funding.
Bottom line is that if we can move toward a more sustainable funding model, it will be good all round. The Eihort licensing extension agreements are out and we’ve had about 40% back so far (all in agreement) - I’ll keep prodding people until we get definitive responses from everyone, or the remaining contributions are minor enough just to rewrite and be done with it.