SteveStreeting.com

Penny Arcade tends to divide a room much like Marmite, but I like both. I was pleased this morning, therefore, to see Torchlight, a game deep within which little gears of my own construction are happily spinning away, featured as the news and comic of the day, and in a very positive fashion.

(image copyright Mike Krahulik & Jerry Holkins, click for full-size from original source)

Needless to say the Runic guys are rather happy about this too, no doubt not only because it’s very cool, but also in a more sober sense that you can’t buy this kind of publicity. Their tweet summed it up: “/dieshappy”.

Woohoo, Torchlight, the new ARPG by Runic Games and using OGRE for rendering, is launching today! Well, strictly speaking the single player game launches today, with an MMO version planned for 2010. Torchlight has been developed in Seattle by a veteran team composed of the designers and leads of projects like Diablo, Diablo II, Mythos, and Fate, so you knew this was going to be good.

Well, Runic were kind enough to send me an advance copy which I played a little yesterday, and boy, is it polished. You can really tell the heritage of this team, it’s immediately fun to play and has really great production values with tons of neat little touches and is something of a visual treat, which considering it was designed to work on low-end hardware too (it has an explicit ‘netbook mode’ for goodness sakes) is no mean feat. A level editor is coming out very soon too – the very same one that Runic used to create all the levels in the game – so that will be a lot of fun to play with too I’m sure.

Obviously we’re very proud that OGRE has been a part of creating this title. Torchlight is available to purchase in about 10 minutes time according to the countown clock, as a digital download from Perfect World, Steam, Direct2Drive and other partners. It’s staggeringly good value, so go get it!

[edit]In case you need more convincing, here’s a nice video review / overview:

As  you know I’ve been reviewing DVCSs lately. I’m taking my time doing real use cases on them, and deliberately not doing the sort of feet-first leap into whatever looks best / most popular on the surface because I don’t particularly want to discover unexpected problems down the track. It’s consuming a lot more time than I expected – I’m writing up my findings and may publish the entire results later on if I can find the time to clean them up and format them better, but for the moment I thought I’d share some experiences with the conversion process of a relatively large, long-lived, multi-branch repository (OGRE) from Subversion to Git and Mercurial, because that’s what I’ve been wrestling with in the last few days. I discovered a bunch of additional issues during this process that did not occur when starting from scratch or doing conversions from more trivial repositories, so I thought it might help others to talk about it.

Source Subversion repositiory specifications

Revisions: 9215 (as of today)
Branches:  9 permanent, 22 temporary / experimental
Size: 375 MB

Also of note is that the source repository is still at Subversion 1.3 – this is because Sourceforge was stuck on this version for a long time and we haven’t upgraded the repository since they started supporting newer versions. We never bothered because it requires locking out the repository while you download the whole thing to a local machine, upgrade it and re-upload it, which is a hassle, especially when you have things to do. In practice the server-side version hasn’t been a major issue since you can still use newer clients with it and svnmerge operates regardless.

General Approach

I rsync the OGRE repository down to a local Linux server several times a week, so that was the source of all my conversions, eliminating most of the network time. I tried to convert the repositories using Windows clients in the first instance, because that was easier to use the latest versions of the tools (my Linux Server is on Ubuntu 8.04 LTS and even with hardy-backports available it’s not as up to date – and for simplicity because this is an important server I stick to the official versions). There is a 1Gb network connection between the machines so it could be pretty speedy.

The principle is that I want to preserve all history, all branches, and all tags. In practice I may actually prune off some branches later on, so that the clone process is quicker, but the base principle is that it should be a lossless conversion in the first instance. Definitely no top-skimming of the trunk like some conversion articles advocate – we have stable branches that must be maintained and regularly have work that we want to keep in experimental branches. In particular, post conversion it must be possible to continue committing to and merging from stable branches.

Git Conversion Experience

I’d previously converted some other, small and fairly simple Subversion repositories using git-svn (less than 500 revisions, and 2-3 branches) and it worked fine. However, when trying it against the considerably more complex OGRE repository I hit problems very quickly. On Windows, using msysGit 1.6.4 the process failed after 1900 revisions, just after doing the automatic repository tidy (git gc). The error message was simply ‘fatal error running git-svn’, even though it had been running exactly that command for the last 1900 revisions. Thinking there might be an msysGit issue here, I switched to the Linux server (git 1.5.4) and tried the same thing. This time it fell over at revision 176 with absolutely no error message. In both cases the repository left behind was corrupt so I could not resume the process.

The other thing I noticed was how long the process took on Windows. 1900 revisions took 5 hours (!) and thus I wasn’t in a hurry to retry the process there. On Linux the process was much faster, as far as it got. It’s worth noting that this is not caused by running across 2 machines – not only do I have a very adequate 1Gb link, Mercurial managed significantly faster conversions using the same topology. msysGit’s git-svn conversion is simply incredibly slow.

At this point I decided to try upgrading the Subversion repository, just in case git-svn hadn’t been tested with older repository versions. My Linux server had svn 1.5 on it, so I upgraded the OGRE repository to that locally and re-ran the git-svn process on the Linux machine (as I say, I wasn’t keen on repeating the glacially slow msysGit conversion). Sure enough, this time all 9200-odd revisions converted fine, in only about 1 hour 40 minutes, or about 15 times faster than doing it on Windows.

So, I may have had a few problems, and being forced to upgrade the repository before converting was a bit of a pain, but at least it worked and was fast (on Linux anyway). After that, I started cloning the repository both on Linux and Windows and tried performing some standard operations.

The first thing that surprised me was that when cloning the converted repository, I could only see the ‘master’ branch on the remote machine. It’s common practice for Git not to create any local branches other than master on clone, but usually you can do ‘git branch -a’ to see all the remote branches that are available, which show up as something like ‘origin/v1-6′ – you can then check them out to local branches. However, no branches other than ‘origin/master’ showed up, even though I knew they’d been converted. It turns out that git-svn converts all branches except master into remote branches in the converted repository, referencing the original Subversion URL – so very much like having cloned from another Git repository. That sort of makes sense, but in the context of a full conversion to a repository that is destined to become the upstream master, isn’t that useful. In practice what you need to do is after the git-svn conversion is complete, git checkout each of the branches that you care about in your converted repository, thus creating local branches in that repository which subsequent cloners will be able to checkout themselves.

So, once I’d figured this out I started to check out different branches to test if it had worked. At first it seemed to, when checking out the first branch (switching from master to v1-6 in a local clone from the conversion). When I came to try to switch back to master however, Git complained that I had modified files in my working directory. WTF? I’d only just checked out the clean copy of the v1-6 branch. But sure enough, git status told me I had 5 modified files. Diffing them showed no changes, and “git reset –hard” returned with no error, but git status still showed these files as modified. Bizarre. A git checkout -f still let me switch, but again after completion a set of other files showed up as modified. Switching back and forth (with -f) a few times revealed that the list of modified files after checkout was different each time. Again worried that this was a Windows thing, I tried checking out on my Linux machine instead (so at that stage the entire process, conversion to checkout, was done on Linux). But no, the same problem occurred – a random selection of 5-7 modified files on clean checkout.

This has raised some serious concerns about using Git for me. Firstly the flaky conversion which requires a bunch of extra steps just to get it to work at all, then the post-conversion bizarre behaviour of thinking files are modified when they’re not. I had none of these problems with smaller repositories, created from scratch or converted, which up until now I’d been using for testing (and Git had been winning me over in fact since it had been working well). But the bottom line is that this process needs to work reliably for the OGRE repository. If it doesn’t, it’s pretty much untenable.

Mercurial Conversion Experience

I started off with the in-built ‘hg convert’ process. It all went smoothly and took about 8 hours, and the resulting repository was mostly fine. However, the default behaviour is to process the revisions in an order which “produces the fewest jumps between branches in the commit log”. In practice, I found that this meant the revision log when reviewing multiple branches was badly jumbled and difficult to use; the use of the ‘–datesort’ option resolved this but increased the conversion time to just under 10 hours (still faster than msysGit but a lot slower than git on Linux).

The guys from BitBucket, who I’d talked to to see if they would offer free unlimited hosting for OGRE since we wouldn’t fit in the default 150MB limit (result was that they were super-friendly and offered not only that but lots of advice), suggested that I try hgsubversion instead. I was initially put off by their website suggesting it wasn’t fit for production use (they’ve removed this statement now), but BitBucket told me that was a little out of date, and in fact the Python project is using it for their conversion, which is obviously of major size. So, I gave it a shot and got some good help from the hgsubversion guys, and the results were great – 1hour 40 minutes from the Windows end (coincidentally the same speed Git managed on the local Linux machine), and the log view was properly ordered right off the bat.

The one remaining issue I had (and this is true of git-svn too) is that all of the branches are open-ended on conversion – that is, no record is made of merges that have been done between branches. That means you would have problems continuing a branch and then merging it, because Mercurial would think it has to merge everything from the point the branch was taken. Neither svnmerge or svn:merge properties are taken into account.

One way to resolve this is to manually create a merge point to close off the branches. The easiest way to do this is:

• Grab the default tip
• Open a command line and define a temporary environment variable “HGMERGE=internal:local”. This means that you want to keep the local files and throw away the other source when doing a merge, which is important for our dummy merge
• hg merge <source_branch> -y
  
• commit – only the .hgtags file should be modified, the rest of the commit is merely metadata alteration to close off the source_branch

Once you’ve done that, your branch is joined back to the trunk and you can carry on as before, any new commits to that branch will merge across cleanly. The only downside of this is that the merge is strictly at the wrong point – if you view the history in the trunk it won’t be technically accurate and you’ll need to use your commit messages as the real guide to the actual merges before the conversion.

A better way to do this would be to record the merges during the conversion, that is for merge commits in Subversion to have 2 parents. So far, none of the conversion tools read svnmerge or svn:merge metadata to implement this, but the standard ‘hg convert’ has an option called ‘–splicemap’ where you can specify merge points to be applied during the conversion. Unfortunately I’ve tried to use this twice so far, and both times it hasn’t worked (just silently done nothing). The documentation for –splicemap is not great so it could be I got the URLs wrong. But anyhow, following 2 failed attempts (20 hours! because this was the standard hg convert with –datesort) I decided I’d try to get a similar bit of functionality working in hgsubversion instead, since that’s much faster (1hr 40m a pop). Right now I’m hacking away on it to try to make this work, so far it’s not but I’ll let you know if I eventually succeed. One of the benefits of Mercurial is that it’s all in Python so it’s very easy to modify, compared to Git which runs all kinds of random scripts and executables, including sh and perl so it’s much more tangled to dig into.

Conclusions, so far

I started my DVCS evaluation very pro-Mercurial and very anti-Git. While working through my detailed use cases, a process which I’ve not quite completed yet, Git has grown on me a great deal, and I discovered a few things about Mercurial which I found a bit limiting at first, but which are mitigated via extensions – Rebase, Queues and Transplant particularly. My recent experience with more complicated, full-scale and imported repositories has once again gone in Mercurial’s favour though, and I saw a nastier side of Git – when it goes wrong, it’s a lot more difficult to figure out why. In contrast when I’ve had my Mercurial conversion crash – and I stress this only happened due to my own screw up, once because it ran overnight when my rsync kicked in and changed the repository under its feet, and a few times when I’ve been experimenting with hacking the Python to get the merges done – the reason has always been clear; a nice Python trace, and the repository was always intact anyhow – in the case of the core hg convert the conversion even restarted from where it left off once I’d fixed it.

If I were to graph my relative opinion of the two over the period I’ve been doing this so far, it would look something like this:

Git totally came up from behind and I was really starting to dig it, until it started freaking out on me with the conversion and I started to try to diagnose why and found it mostly unhelpful. Again I stress I’m not done with my tests yet, but I’m perhaps 75% of the way through now and the conversion problems I’ve had with Git in the last few days don’t look good. Bazaar, I’m afraid, is no longer likely to be part of the evaluation – it takes a long time to do these evaluations properly rather than just trivially, and our survey has indicated that it is the least commonly used among our community by a very large margin, so I’m focussing on the ones more users are likely to already be comfortable with.

The evaluation process continues…

Before 2009, I’d never set foot in Germany before; not for any particular reason, I just hadn’t gotten around to it yet. However, thanks to gracious invitations to conferences I’ve now been twice. In May I went to Stuttgart for FMX, and last week I went to Munich for Qt Developer Days.

It was an enjoyable conference, as always the best part is just meeting other delegates, the sessions themselves are merely the icing on the cake. I shared my presenting slot (in which I showed a couple of applications that use Qt and Ogre together) with two other open source veterans from projects which I have a huge amount of respect for: Bill Hoffman, CTO at Kitware and the founder of CMake (which of course we use in Ogre now, so it was great that I had chance to have quite a few discussions with Bill), and Jean-Baptiste Kempf, Chairman of VideoLAN which is of course in charge of the excellent VLC.

It was also nice (not to mention flattering and somewhat humbling) to have random people I’ve never met before spontaneously say nice things about Ogre. One of the major curiosities of open source is that you never really know quite how many people have encountered & used your software; you get a sampling of that through your community forums etc, but it’s also clear that that only represents a portion of your user base. On the day I was wearing my Ogre T-shirt I had a number of people who were more peripherally involved in the community but who had had a good experience with Ogre, and were more than happy to tell me about it. Definitely a good feeling.

Perhaps most surprising of all though was getting a sizable donation to Ogre in person from a community member while I was there (I won’t mention who just in case he’d rather not be identified, he can post in the comments if he’s happy to). We had what I thought was a theoretical discussion at one of the dinners about how much we get charged by PayPal for donations, and I’d said that although it’s undesirable, any kind of electronic payment mechanism has a cost (merchant accounts, bank transfers all come with some kind of charge). I jokingly said that the way you’d avoid the most charges would be mailing cash in an envelope, although that had it’s own risks. I thought nothing more of it, until I saw him the next day when he presented me with an envelope with a donation in it! Way more than I expected too, enough to push him straight to a Platinum sponsor. Turns out he’d just got his deposit back on a flat he had been renting, and decided to donate that to us in the absence of any code contributions, since Ogre had helped him at university and subsequently in getting a job. I have to admit, I was a little lost for words at that! His donation will definitely help cover the server running costs in the coming few months.

Back to the conference subject, Qt, the conference reinforced my opinion that it’s the best cross-platform UI system out there for C++ developers. It was great to see the range of applications that were being developed on it these days, including a coffee machine which was serving custom beverages in the dining area via a Qt interface. Obviously the Nokia acquisition has meant that they’re keen to move into more dynamic, touch-based interfaces too now, which will obviously power new phones in more interesting ways, but it was clear that they remained committed to a huge range of application targets. Well, except iPhone anyway, that was definitely the elephant in the room – occasionally mentioned but mostly avoided

Obviously Qt’s switch to LGPL this year will have a huge impact on adoption rates. One of the things that had concerned me though is that there’s a clause in the commercial license for Qt that requires you to decide between using the commercial license and the LGPL before you start developing. The reason given for this is that Qt is licensed on a per-developer basis, so if you could wait until deployment to choose the commercial license, you could scale back your team and pay less than you really should have done, which is why you have to decide up-front. I could understand this argument, but in my experience, perfect foresight is impractical and conditions can often change, so making a once-and-for-all choice before a line of code is written did not seem realistic in some cases. Also since the principle was that the entire team must use the same license, I was wondering about the practical implications of say, a commercial outfit leveraging some pre-written code by people using the LGPL version (such as QtOgre). Qt want to encourage greater community involvement (which was the reason for me being invited to the conference after all), so not allowing this seemed to go against the kind of broader adoption they were chasing.

To try to answer these questions, I went to the legal presentation and put this question to the speaker afterwards. Luckily, she mostly allayed my fears on these two issues. On the ‘circumstances change’ issue, the principle must remain that, because of the per-developer licensing, you should make your decision up-front, but if for whatever reason, and in good faith, conditions change (such as suddenly having to target a platform where LGPL is not practical), then some kind of agreement can be reached, such as by paying commercial fees for all developers historically on that project so it comes out the same as if you had opted for the commercial license originally. In addition, she didn’t think there would be a problem with community code re-use for commercial licensees provided this was mentioned during the commercial licensing process; she accepted that greater adoption is what they want, and community development inherently complicates the previous assumption that one team will be responsible for absolutely every aspect end-to-end.

So, a good conference overall. Now, I’m back to continue work on Ogre 1.7.

I’ve been pushing quite hard to get this done before I head off to Qt DevDays next week, and luckily it all came together in the last few days:

Some of the notable back-of-the-box (if there was a box) items:

• Upgrade to SpeedTree v5 – supporting all the great new features. See the SpeedTree site for more details on this release.
• More lighting options – Ambient Occlusion, Ambient Contrast, Specular Lighting, Transmission Lighting, Global Light Scalar, HDR.
• Improved LOD fading – LOD transitions are now implemented via less perceptible techniques thanks to SpeedTree v5.
• Dynamic soft self-shadowing via depth shadowmaps – (was also demonstrated on SpeedTree 4.2 in OgreSpeedTree 1.1.0)
• Improved alpha to coverage support – alpha to coverage rendering has been significantly improved and is now enabled by default (you must render with at least 2xMSAA to see the effect), resulting in significantly smoother foliage edges and smoother fade transitions.

I’m just really, really glad to have got this done. SpeedTree v5 was quite a big change to handle, but hugely worthwhile, the guys at IDV have done a great job. I made the choice from v1 to do a ‘deep’ integration (ie all objects are first-party Ogre objects, everything uses Materials and RenderQueues properly, etc), which made the transition considerably more time consuming than using the higher level pass-through interfaces, so this took some effort. I like it this way though, it means it’s a ‘real’ integration as far as I’m concerned, not just a simplistic bolt-on.

There’s still more to do, not least of which a demo that’s more interesting than just a technical performance test But at least I can tick off one of the major things on my TODO now. I’d been putting in quite a few extra hours to try to handle this, and my Qt presentation preparation, and Ogre, and consultancy, and it has taken its toll a bit – I’ve been spending more time at the desk and think that was a contributing factor to me straining my back at the gym last weekend. Hopefully that’s just temporary, I’ve been a lot better these last few months.

Hope you like the shots anyway. I know, I’m pimping them all over the place in my Twitter, here and on the Ogre site, but a guys gotta make a living, right?

I’ve been involved in open source for a long time – probably what might be considered a ‘generation’ in this industry. I was a fan of open source before I even knew the term existed – during my formative coding years in the early 90’s I was always releasing code for free and encouraging people to tell me why it sucked, and doing the same for them. Of course, most of the discussion went on over FidoNet, BBS-relayed emails, the very early (pre-WWW) internet and code on FTP sites, but the principle was much the same.

It was only when I needed a host for an increasingly large 3D engine I’d been writing in 2000 (which later became the earliest version of OGRE) that I discovered Sourceforge and open-source licenses. At that stage, I didn’t really care much about the minutae and just went with what seemed the most popular choice – the GPL. Fairly soon after this was revised to LGPL because it was clear GPL wasn’t a flexible choice for a library. Over the years I formed the opinion that the LGPL was a good balance for what I was doing – it allowed use in proprietary applications but required that modifications to the open-source part were passed on; and this also, I thought, encouraged contributions back to the core since it’s simpler to have your changes promoted. The downside was that the license is pretty long and full of legalese, and frequently needed explaining & clarifying. Still, I considered that a necessary cost of a license that ‘protected the interests’ of the open source project. I had considered more permissive licenses like zlib and MIT to be too weak -  giving everything away with no conditions encouraging reciprocal contributions or participation.

I only really started to seriously question this long-held opinion recently, shortly after I had to deal with a license violation issue. The details of this are irrelevant (and will not be discussed here), the important thing is that after all the dust had settled, it really made me sit down and take a long, hard look at what open source licenses actually achieve in practice, and which bits I felt were the most important for sustaining my projects.

Purist free software advocates subscribe to the view that all software should be ‘free’ (as in speech), which is, when you boil it down, a philosophical principle. While the intention is often genuinely altruistic – enablement for all developers and users of software – in a practical sense it is also a highly prescriptive, black-and-white approach. Free software licenses like the GPL restrict as much as they enable, on the basis that not to do so would deny future users of the flexibility to alter the software. It’s an understandable point of view, if very hacker-oriented, but it’s not one that I personally sign up to. I’m not involved in open source as a ‘movement’, in order to ensure that all users have access to all the source code of every application they use. Instead, I’m involved in open source because I think it’s a damn good way to get good software written for the benefit of the many, particularly infrastructural elements that can then be used to innovate faster, by smaller teams, to challenge incumbents, and generally shake things up. I’m a firm believer that open source and proprietary software can be good bedfellows, and that each can benefit the other – open source providing solid foundations for proprietary innovation, and proprietary software contributing to the open source projects underpinning it, in terms of code and funds. In that context, the restrictive elements of the LGPL seemed well-placed – the LGPL allows proprietary use, but modifications had to be passed on.

But, after much consideration and hard self-questioning in recent weeks I realised something important. Of all the people that have contributed to OGRE over the years, I can’t think of a single good code contribution that has come about because the license conditions encouraged it. All of the people who have made a significant positive contribution to OGRE have done so because they chose to do so. Either because they wanted to do it for fun, or because they saw that it was in their own interests as users, they sent patches in, created new add-ons, joined the team etc. None of these people required any coersion from a license to do what they did, and all the better for it – because they chose to do it, they tended to be more forthcoming in terms of adhering to standards, answering questions about their contribution, and generally participating in the community rather than just doing a minimal code-dump.

The other thing I considered was the impact of someone modifying OGRE and not publishing their changes. On the one hand, this is a potential for a lost contribution, but in practice the LGPL conditions are only to pass on the source to others, not to contribute back to the core project (which requires a contributor agreement anyway), and as mentioned we’ve already established that the best contributions are from willing sources anyway. Then comes the principle of whether it matters that the end user is not guaranteed to see the modified source of OGRE – free software principles say this is not right, but I have to say that in practice, most end users don’t care and in practice it’s of little consequence for OGRE. Obviously many people in the free software camp would disagree strongly with this. It’s more of an issue in enterprise apps where a customer who receives a significantly customised application from a vendor, and needs to keep maintenance in mind; IMO regardless of whether the software is based on open source the customer should be insisting their vendor delivers source anyway – that’s what contracts are for. But I digress.

So, after much thought I concluded that the most useful pay-backs to an open source project, and thus its community, from a user (in my opinion) were:

1. Code & documentation contributions – which based on my experience come from voluntary sources
2. Community participation – forum support, bug reports etc
3. Publicity.

The ‘restrictive’ elements of the LGPL (and GPL), to which so much confusing license text is dedicated, didn’t seem to contribute to any of those except number 3, and then really just as a side-effect.

It was at this point that I realised that my previous opinions about permissive licenses not providing enough safeguards against exploitation for an open source project were off-base. In practice, open source projects don’t really need protection, because their best contributors are going to be there regardless (yes, I realise the GPL provides more protection to end users who want to get at the source code, that’s not what I’m considering here). ‘Freeloaders’ – people who use or modify the open source project for their own ends but give no code or community contribution back – are always going to exist; even under the GPL it’s easy to freeload, if you make your money from hosting services for example, and thus license choice has little impact on the scale (if not the nature) of the freeloading. Besides the annoyance of ‘that guy took my work and made some money out of it’ – which you have to accept as an inevitable outcome of going open source, so stick to making proprietary software if that bugs you – freeloaders have little negative effect on an open source project, and actually their use can contribute positively to item 3 (publicity). The key is to recognise that in practice you can really just ignore freeloaders, and instead concentrate on maximising the positive contributors in your community.

So, if we acknowledge that the people whose contributions we actually want are those who contribute voluntarily, regardless of license, we quickly come to the conclusion that all that really matters is the size of the community. It’s a fair assumption that for a given project there is a relatively stable percentage of users who will choose to contribute back (the percentage itself varies per project, but is fairly stable per project in my experience), therefore the easiest way to increase your contributors is to just increase your user base. Forget about trying to coerce people into being ‘good’ members of the community, just trust that the percentage will be there and will track your overall numbers.

One way in which to attract more users is to make the licensing simpler and more easier to understand. Programmers hate legalese, and a simple, clear license is bound to be more attractive than our LGPL (with static link exclusion), plus OUL option. It’s for this reason that from OGRE 1.7 we’re switching to the MIT License.

I’d like to thank Matt Asay for his post on the subject, which really jump-started my thought processes on this subject in the last few weeks. This whole process has been a reminder to me that it’s always good to re-examine your previous assumptions and opinions in the light of practical experience, and to be prepared to reach different conclusions than you did before. The whole open source landscape is constantly changing and maturing, why should those of us immersed in it be any different?

Hell yes. It’s nice to come back from holiday to more eye-candy from leading OGRE-based projects, and this one is looking very nice indeed. Congrats to Travis and the rest of the team at Runic, on the PAX ‘09 coverage and also the recent Gamasutra article which I read on my phone at breakfast one day while on holiday (much to my wife’s annoyance ).

I’m busy, again. A ton of things just bunched up towards the end of the month, and I’m on-site with a customer in Cambridge some of next week, so I’m keeping my head down a little right now. Here’s a news-blast though.

I love Ubuntu server

I’ve been setting up my new server. I’ve probably said this before, but for servers, Linux rocks. I’m ambivalent about Linux on the desktop, where I believe consistency and usability are more important (the Mac floats my boat the most there, and Windows if only because of MSVC++), but for a server Linux really brings great things to the table. Rock solid server apps, ridiculously good performance for the hardware (a mere Intel Atom 330, goes like a greased whippet), easy and most importantly infrequent maintenance. When it comes to distros, I loved Debian for its sensible defaults and great package management, but Ubuntu server takes that and makes it even better.  The added bonus of a LTS version that I know will be supported with security updates until at least 2013 is welcome too, because I like to set these things up and mostly forget about them.

I’m glad Ubuntu’s default mail/IMAP servers are Postfix and Dovecot respectively – they’re just ridiculously easy to set up. I’d been using Exim4 on my Debian box, which was the default at the time, and learned to dislike it because of the over-complex configuration (I’d been used to Postfix when I ran a Gentoo server before that). I’m also planning on trying out Bacula as a replacement for my manual backup scripts this time around.

PSUs hate me

One of my jobs this week was building a machine around some customer hardware which I was testing an abberation on. Having built it, I realised I didn’t have a spare PSU rated highly enough, so I ‘borrowed’ one out of another GPU test machine I had. It ran all day, then decided to die – this just 2 weeks since my server’s PSU died and needed to be replaced. It’s just bad luck – my decent APC UPS should be providing ample power regulation. So, I’ve ordered 2 new PSUs to make sure I have enough stocked in future!

Excitement is infectious

I was happy to show off some shots of my new & improved core terrain system for Ogre, which isn’t entirely done yet but was usable enough to get some nice shots out of. I knew I had to move on to some other work for a while but I was still pleased to be able to show off some initial eye candy (which BTW, is still very early – I haven’t finished yet by any means). I was glad to get some positive & constructive feedback, but of course now people are rushing off an including it in their projects, since all the code is public in svn. Despite my slapping a big red warning sticker on it saying ‘handle with care – volatile material’  and that they shouldn’t assume it works properly yet, people are hacking on it already, with some nice results I have to say (such as Ogitor integration) – but of course with many questions and issues. Such is the nature of open source – it’s a blessing that you get instant, voluminous feedback, and it’s a curse that you get instant, voluminous feedback I hope to get more time to deal with the fallout from that if not next week (because of my travels), then the week after.

The OGRE Patch Mountain

Our community is always active, and it’s great to get patches. I do have a quite high validation standard for the core though, and processing patches can often take a fair bit of time. I try to spend a few hours per week doing this, but mostly that’s only just enough to keep the level static, rather than reducing the backlog, and it still spikes up sometimes (as it has this week) – that’s because even if I get an afternoon on it, to review and test things properly can eat that up very fast.

If there are any experienced members of the Ogre community who would like to assist me with keeping the patch mountain down to a small hillock in future, and are willing to adhere to our high standards of review, please contact me at sinbad AT ogre3d DOT org.

iPhone 3G / 3GS port coming soon

We’ve had a fledgeling GLES rendersystem around for a while, and obviously the iPhone / iPod touch are the highest-profile targets of that. I’d been intending to have a go at it later in the summer, but masterfalcon on the forums has beaten me to it and already has it running (with a few small issues remaining to be ironed out) on the 3G and 3GS. There should be a public release of that in the relatively near future.

3D Web Browsing With OGRE

I love this video. Nice work princeofcode (aka ajs15822)

That’ll do for now I think.

Bah. I’ve used svnmerge for ages to meet my Subversion merging needs, but a change around property management in Subversion 1.6 makes it no longer work anymore.

Yes, I know Subversion has had its own merging features since 1.5, but I still use svnmerge because it works well, it generates me detailed merge commit messages, and it doesn’t require me to upgrade all my repositories to use it. Sourceforge was on Subversion 1.3 for ages so this was very useful, but even now they’re on 1.5 it’s a PITA to upgrade your repository, because you have to re-upload the whole thing to do it (I already have a local rsync copy anyway, but you have to locally upgrade it and re-upload). Obviously this is not the kind of thing you want to be doing when development is going on, and especially not when you have Summer of Code students getting on with things.

The answer for me was to downgrade back to Subversion 1.5 because that still works both ways. I’ll maybe start using Subversion’s in-built merging once I’m confident it’s no longer less functional than svnmerge, and when/if I can find a suitable window to take the repository offline for everyone for a few hours. Right now, it’s just not worth the hassle.

[edit]Aha – an alternative is to use this patch to svnmerge to make it work with svn 1.6’s altered error reporting behaviour when deleting a property. For some reason the svn 1.6.2 official Windows distribution didn’t seem to have this in the contrib folder.

I can’t remember who made the assertion / joke that if you looked through an infinitely powerful telescope you’d end up seeing the back of of your own head, but I was reminded of that by a certain event today. In the last couple of years I’ve often Googled for a particular subject and ended up with the top hits pointing me back at one of my own posts in the OGRE Forum or on my blog, in a weird self-citing manner. In the worst cases, these posts answer or clarify my own current question, because a thought process I’d had a few years before, and then forgotten about, can often be useful. It’s like having a stack of your old notebooks in the cloud! Or an archived clone of yourself pointing out how age is making you stupid.

The other weird experience is when you download or otherwise get hold of a piece of software, and unexpectedly find that it uses your own code. I’m sure this is common in open source circles, because users of open source don’t have to tell you when they use your code, but nevertheless it’s still an odd experience. It’s especially nice when you like that piece of software – this happened to me today with PixPlant 2.

As I mentioned yesterday, I was reviewing tools for normal/displacement/specular map generation from reference sources, and I’d been evaluating CrazyBump and ShaderMap Pro. Evak in the OGRE Forums suggested I try PixPlant2 because he liked it. So I did, and I was impressed – the texture generation seemed as good as CrazyBump, but it’s cheaper ($175 rather than $299), and it also includes tools for creating tileable textures from original sources, detecting repeating patterns, straightening things, and blending the edges for you.

So, I was already leaning towards this purchasing PixPlant2, but then as I was browsing for textures, I noticed that the PixPlant2 application folder had some familiar files in it – such as OgreMain.dll, and rather familiar material files in the media folders! Checking the docs, sure enough OGRE was credited as a dependency. The application I ended up gravitating towards included software that I wrote!

To cap it all off, they’ve been very nice and offered me a free copy, so my normal map generation needs are entirely satisfied, for far less than I was expecting to pay. It’s not often things work out quite so well!