SteveStreeting.com

The trouble with developing a project which is not only cross-platform, but is old mature & wizened enough to have users dotted across a whole history of incompatible versions of the same tool, is that you end up having to maintain a ton of project files. Linux makefiles, 3+ versions of Visual Studio, Code::Blocks, XCode, Eclipse - it all gets a little unmanageable. I’ve been meaning to look at cross-environment build managers like CMake for a long time, but the time investment required to port & test, let alone learning how to do it, was daunting enough that it never seemed to make it to the top of my TODO. One of the problems was that despite CMake being well respected (projects like KDE and VTK use it), the documentation is a little impenetrable - most of the online stuff being a pretty dry reference that’s difficult to get started with. There is a book, but it’s hard to get hold of and the prices are just crazy even for a technical book - Amazon sellers were asking 70 quid for it used! As such I was never sure if it was going to work out and didn’t really have the spare time to try it.

But, as is so often the case, here’s where the open source community steps in and helps out. CABAListic (aka Holger) has been working really hard not only to get OGRE building with CMake, but doing a damn fine job of it too, with the kind of attention to detail that really makes all the difference. He’s done all the heavy lifting, and by looking at his work I’ve been able to get up to speed fairly quickly, both with the principles and some good practices, and I’m really starting to like CMake a great deal. Right now, it exists in parallel with the other builds in trunk, but relatively soon we’re expecting to make CMake the primary build system - I’m even skipping updating the old project files for the new paging component I’m writing right now and going straight to CMake, and it’s really very nice to use. Sure, you have to edit text files, but they’re easy once you have a pattern defined, and the way CMake checks them before building & refreshes the generated configuration is pretty slick. And I certainly won’t miss poring through page after page of project settings in VS, sometimes for each build configuration, to manually tweak options for multiple projects.

As well as not needing to update so many project files (the exception is XCode which will be separate for now, since I’ve yet to figure out if CMake can do everything I need it to, given the oddities of frameworks and universal binaries on OS X), it’s nice to be able to give people an interface to creating custom OGRE builds (e.g. enabling threading modes, double-precision, static linking etc), and the ability to build them into external directory structures from one source release. Having the equivalent of a ‘configure’ step (as seen on Linux) everywhere is also very nice, for performing sanity checks on dependencies and the like. Also, the fact that the source build collates the final result in a separate location means we can finally unify the directory structures as viewed by 3rd parties between the SDK and source releases, which will smooth things for people moving between the two or publishing add-ons for people using both.  We can also provide CMake files to make it easier for people to locate OGRE when they want to build against it.

Doing something as major as replacing the entire build mechanism for a project is pretty significant, but I can already see the benefits. Huge thanks to CABAListic for getting this rolling!

Important: the subject matter and parties involved with this legal issue are deliberately not mentioned here; if you are aware of their identities, I ask you not to mention them publicly here in comments, or anywhere else.

As some of you are already aware, over the last few months there has been an ongoing legal issue with a 3rd party having allegedly used OGRE code without respecting the license conditions. I hate getting involved in legal disputes, there are so many more useful things to do with time, money, and emotional energy, but nevertheless as custodian of OGRE it falls to me and my company to take charge of situations like this, however reluctantly.

Happily, the issue was finally resolved today. Details are confidential (so don’t ask), but what I can say is that a satisfactory resolution was achieved, and without me having to remortgage to fund the legal bill. The Free Software Foundation Europe were very instrumental in advising me on the case, and directing me towards good legal representation, so I really can’t thank them enough for their help; but I hope that the donation I’ll be making will go some way to repaying my debt to them.

On the whole though, I’m just glad it’s all out of the way, it’s one less distraction

Obviously I’ve been doing a lot of web work this week, getting ogre3d.org back to how I want it. In a lot of cases that’s meant changing the software wholesale - something you really don’t want to be rushed into, but in this case I had no choice.

It’s been one of those weeks where I feel like I’m desperately doing a headlong sprint, only to run into an invisible brick wall two feet from the finish line. A number of the pieces of software I’ve tried using have been in the ‘almost, but not quite’ bracket. We used Gallery for years for example, but had never upgraded past version 1.5. Given that I’m making sure we’re not languishing on any old versions any more, I obviously picked up Gallery 2.3, spent a fair amount of time re-importing all of our galleries with the metadata attached, integrating it with WordPress via WPG2, and styling / configuring it the way I wanted, only to find that when I put it on the live site, the memory usage was completely unacceptable, even with all non-essential plugins disabled and full optimisation turned on. We’re a reasonably high-traffic site; nothing compared to some of course, but still higher than a lot of places), and it was clear fairly quickly that Gallery 2.x was going to be a dead weight - literally as soon as I enabled it the server’s available memory would plummet like a stone. I wish I had a representative performance testing set up to try these things out beforehand - sometimes I miss having the resources of a larger organisation to tap into in cases like this!

Rather annoyed at the time I’d wasted, I then tried NextGEN Gallery for a little while, but found that inadequate too. Again it was almost good enough, but unfortunately the navigation between albums was not very good, and would require some modding to make it work well. And that was even before I knew how it would perform - I suspected better than Gallery, since it’s a lot less functional, but still, it was an unknown, and I didn’t want to spend time fixing it up the way I wanted (like I did with Gallery) only to find I have to toss it out.

So finally, I settled on hosting our gallery on Flickr instead of self-hosting, and exposing them on the main site via Slickr Gallery. Slickr has a really nice navigation mode and looks very slick (it’s aptly named), and using Flickr for hosting obviously has bandwidth and performance advantages.

The one problem with this solution though was that the lightbox that Slickr uses is a bit weak. It can only handle a fixed-size window (although it will resize to the aspect ratio of the picture), which means the image is almost always too small on a decent sized screen. You can increase the size, but then it’s too large for smaller screens and you have to scroll, so it just ends up feeling unsatisfactory. So, instead I wanted to use Shadowbox.js, which resizes dynamically to the client area of the web page, which is ideal. Unfortunately Slickr doesn’t have an option to change the lightbox, and there wasn’t much information online about it. So, after some tinkering, I discovered how to do it:

1. You should have Slickr, Lightbox-2 and Shadowbox.js installed.
2. In wp-content/plugins/slickr-gallery/slickr/slickr.php, replace all instances of ‘lightbox’ with ’shadowbox’
3. In wp-content/plugins/slickr-gallery/slickr/slickr.js, replace “initLightbox();” with “Shadowbox.setup();”

That’s it.  It’s fairly simple, but it took me a bit of trial and error to get completely right even though the end result was trivial (Shadowbox’s code is a little hard to read). I originally made more changes thinking I had to do more than this, but ended up breaking things in IE, or in certain navigation cases. This simple change seems to work in all reasonable cases (IE7, FF3, Opera9, haven’t tried Safari yet). You could probably make more changes to remove the Lightbox-2 dependency entirely (disabling the plugin breaks Slickr still), but I have other things to be getting on with for now. Like uploading and tagging all the old screenshots!

Edit: here’s the result. Yes, I know the page title says ‘Page Not Found’ (something to do with the way Slickr redirects page entries), will try to figure out how to fix that later.

So, now that I have basic service back at my newly reinstalled server, it’s time to start re-downloading clean packages for the software that makes up ogre3d.org. Imagine my surprise when I went to phpBB.com and saw this:

Looks like I’m not the only one having a bad week

Current priority is getting the forums back online, watch that space.

Great. I got a call from the provider of the server on which ogre3d.org runs today saying that the server had been compromised, and that they were going to have to resinstall it. Seems that an unpatched flaw in a Joomla! component allowed a backdoor to be created which was spamming.

My fault - I should have patched Joomla! more often. But, I’m busy. What’s annoying is that even though I can 100% guarantee that the compromise could not have extended beyond a certain defined scope, since SELinux prevents that at the kernel level, the host has insisted on taking a tough line on this - their unshakeable policy says that a completely fresh reinstall is required, and it has to be done today. I can understand that, but I see now it’s been pointless for me to use SELinux in this case then. They’ll charge me extra for the reinstall too The only benefit is that at least while I’m doing it I can hop onto RHEL 5 which will mean MediaWiki can be upgraded at last.

This is going to cause some major downtime on ogre3d.org, firstly for the physical reinstall, and secondly for the time it takes me to get the ton of application software reinstalled. First priority will go to getting the forums running again, obviously. I’ll have to decide what to put on the main site page - it’ll be a placeholder at first. It’s very likely to be a few days before everything is completely back to normal again.

Don’t expect me to be doing much except getting that sorted out for a little while. I’m expecting to be stressed. Once I’ve fought these particular fires and got things on an even keel again, I’m considering moving host again since clearly the ‘premium’ support that I pay for is actually a pretty blunt instrument (you can say  you told me so now).

Update: ogre3d.org is offline as of now for reconstruction. Subversion and downloads are still available on Sourceforge, once the server has basic functionality back I’ll put up a placeholder letting people know what’s going on.

Allegedly, as reported by Yahoo, MTV intend on having 5000 tracks available in Rock Band by the close of 2009. That’s right, 5000. Surely they can’t be serious? (*insert Airplane joke here*). That’s a completely bonkers number - sure, they blew through the 500 mark ahead of schedule and are up around 600 now, but that would average 400 new tracks every month. I can’t help but conclude that this was a misquote, marketing bravado, or a mistake. But who knows? Here’s what was said:

The “Rock Band” and “Guitar Hero” franchises will keep leading the way in music sales. “Rock Band,” which has been the most aggressive to date, offers new music every week, maintains a selection of more than 500 songs on its platform and has sold an average of four songs per user. MTV said it plans to increase the number of available songs this year to as many as 5,000.

Four tracks average per user? Hmm, guess that makes me an atypical user then, I have over 70

Edit: Yep, misquote by the journo. I bet a whole bunch of note charters at Harmonix are breathing a sigh of relief!

I’m a bit of a grump when it comes to a lot of the Web 2.0 startups of recent years. I still dislike Facebook - originally it was just an in-principle reaction based on their rather irritating child-CEO and his ability to attract vast amounts of investment based on a business plan made entirely of arm-waving and wet tissue paper, but now having used it for a while, I dislike it on its own merits. Even with my relatively small friends list it seems dedicated to the task of swamping me with as much useless information, stupid applications, and other such nonsense as it possibly can. Occasionally there will be a nugget of something in there, but it’s drowned in so much crap it’s hardly worth the effort to filter it. Sure, it allows me to keep up with the activities of ‘friends’, but in a much more shallow and sterile way than actually meeting them or talking on the phone/skype/in-game chat. Anyone who thinks Facebook is a good way to develop actual friendships is deluded. As a supplement for real-world or voice meetings (e.g. arranging get-togethers, sharing photos etc), I can see the utility, but I attest that it’s really not even the most efficient way to do that, what with all the junk on there. So generally, pass.

LinkedIn I like, because it sticks to the point. Every business person needs their Rolodex, and that’s what LinkedIn is - a business tool. It doesn’t claim to be central to your life, or that you can build / sustain real friendships there or other such twaddle. The focussed nature of it means there’s less noise, so it’s easier and quicker to get what you need. Contacts there are not personal, they’re a reference for it you’re looking for work, a contractor, a partner etc - which is appropriate for the medium in a way that personal relationships are not. I don’t feel the need to constantly update it or prod my contacts, or keep track of all their statuses; it just does what I need and gets out of the way.

I’ve avoided Twitter because again, it just looked like a way to pour time down the toilet. It seemed like blogging without the in-built filter that says ‘is this worth blogging about’?. Besides, I already actively participate in forums, mailing lists, blogs etc - I really didn’t feel I needed yet another dimension to have online discussions in. And I don’t really want to tell the entire world what I’m doing at any one time - I strain out the small number of actually interesting things already and post them up on this blog, and the rest of the minutae rapidly decreases in value the more degrees of separation away from me you get; so in fact real-world communication works far better as a filtering mechanism for that. Clay Shirky described why information filters in the real world have historically done a much better job with these kinds of things.

However, I decided to open a Twitter account anyway, but only for a very narrow subset of ‘what I’m doing’ - that is, what I’m currently doing in core Ogre. Obviously I blog about that sometimes, and I post in the forum for things that people need to know about, and keep the roadmap on the wiki up to date, but sometimes there are low-level things that don’t warrant that, but people are still interested in. I used to use IRC for this, but over the years drifted away from it, as it was too time consuming to loiter in channels for extended periods looking for topics in real-time, and people just bombarded me with questions all the time anyway so it was a little too much like hard work. So, for things that don’t warrant a blog or forum posting, I’m going to try to keep Twitter informed. There’s also a widget on the right there.

The focus will be kept deliberately very tight - so nothing about my other work commitments, personal life etc are going on there. If any of those things merit comment, they’ll get a more considered entry in my blog. Consider this a sort of ‘pre commit log’ of sorts! If I continue with it, anyway…

Every day I get at least one email from people asking me to manually activate their OGRE forum account because they answered the COPPA question wrong:

Honestly, how hard is this to answer correctly?

*sigh*

NimbleBit, those cheeky purveyors of nimble bits, recently released a new teaser-style video of their Mario Kart-esque game ZeroGear (based on Ogre of course), which I thought I’d share with you in case you hadn’t already seen it on the forums. I think you’ll agree it’s looking very slick.

They’re currently in beta and are looking to get a little more publicity as they line up for their next beta drop, so if you like what you see, be sure to spread the word!

I was quite gratified to read this post on Wolfgang Engel’s blog, in which he refers to some other posts discussing the recommended categorisation & nomenclature for the various stages / structures of scene rendering. If you read it and you’re an OGRE user, you’ll find them all rather familiar concepts, because OGRE has been based around these principles for years

“SpatialGraph: used for finding out what is visible and should be drawn. Should make culling fast”

That’s our SceneManager and it’s subclasses. One of the core tenets of my design for OGRE from day 1 was that the mechanism for performing fast culling should be customisable based on the scene type independent of the scene content, and therefore be able to be divorced from, but derived from, the primary scene structure (the SceneTree, see below). While other engines either tended to hardcode their culling strategy, or retrofit it into the ‘main’ scene graph as custom node types (which tends to encourage the ‘parallel inheritance hierarchy’ anti-pattern, because you mostly specialise nodes for other reasons too, and also makes macro-level decisions difficult), specialised SceneManagers build separate and derived culling structures which are entirely theirs to own.

“SceneTree: used for hierarchical animations, e.g. skeletal animation or a sword held in a character’s hand”

That’s easy - that’s our Node and all it’s subclasses like Bone and SceneNode. It’s rare for an engine not to have this concept, but too many engines make these nodes do too much IMO - like holding material state, being derived for custom culling routines, etc. I’ve always liked the idea of keeping the focus of a class tight (the principle of cohesion), it tends to work better long-term.

“RenderQueue: is filled by the SpatialGraph. Renders visible stuff fast. It sorts sub arrays per key, each key holding data such as depth, shaderID etc.”

Yep, we have exactly this structure and unsurprisingly it’s called RenderQueue We have the concept of queue groups for user-customisable ‘fire breaks’ between rendered objects, and the ability to sort by pass, by distance, or via a custom RenderQueueInvocationSequence.

It’s nice to read posts like this - makes me think we’ve been doing things the right way