SteveStreeting.com

I don’t blog as much as I used to, for reasons which are somewhat relevant to this post – rather you can usually find fragments of my consciousness floating around the Twittersphere instead, since its enforced brevity requires considerably less of my time to populate. Maybe I’m old fashioned, but I believe that if you’re going to write a blog post about something, you should probably make sure it’s written in a half-decent way, and that’s fairly time consuming, particularly when you’re ever so slightly anal about language as I am. Nevertheless, the receding silhouette of 2010 is a worthy enough subject to invest a bit of time in, so here I am, verging once again on being dangerously verbose. I guess the post should have started with a warning banner or something – but if you’re a repeat visitor, you know the score.

So, 2010. I started this year with three significant and intersecting goals: to significantly simplify my working life, to reduce my stress levels, and to spend more time in the company of my own creativity by working on my own projects. The reason was this: by the end of 2009 several factors had led me to a situation where I had my fingers in too many pies, too many balls in the air, too many – well, you get the picture. The thing with being a consultant / contractor is that work comes in with a very uncertain frequency; projects have a tendency to ‘bunch up’, and because of the inevitable lean periods in between you don’t really want to say ‘no’ to anything. The economic situation since late 2008 also led to projects being noticeably more cost-sensitive, meaning more juggling required to fill out a coherent schedule than it had been before that. All this could get quite stressful, and with a back problem to nurse and an open source project to run on top, things weren’t that much fun at times. So my resolution for 2010 was to stop just trying to make the status quo work, and instead to do things my way – to ditch smaller pieces of work, to only get involved in larger projects that I found personally interesting, and to spend the rest of the time investing in my own projects instead. Fundamentally it was about taking control back.

At that time I didn’t plan to retire from OGRE, but in hindsight I think subconsciously I’d made the decision already, I just hadn’t admitted it to myself yet. I’m extremely proud of what I managed to accomplish with OGRE with the help of countless contributors, and was sad to put that role aside after 10 years, but in many ways it had become a rod for my own back. Not only was it a hugely time consuming, 24/7 job to look after, but there was an inherent assumption, from others and from myself, that projects I chose to work on would naturally be associated with OGRE in some way. That was quite a pair of blinkers to have on, even if I didn’t know I was wearing them. If success with a product or in a subject area has a downside, it’s probably that it can become a gravity well which resists you exploring other areas of interest. So while I’m sad to have retired from OGRE and look back on my time on the project with a great deal of fondness and pride, I can’t deny that I don’t regret casting off and returning to the open seas, even if the security of the harbour was comforting. Ok, I’m done with the sailing metaphors now.

One unexplored continent (damn!) which was calling me siren-like (ok, I really am done now) was native Mac development. I bought my first Mac in 2007, initially only to support OGRE on it (before masterfalcon came along with his superlative Apple-fu), but I’ve since been converted to a fully card-carrying Mac nutcase. So, I decided to finally go all-in and learn Objective-C and Cocoa, and make my own Mac-only tool, which of course turned out to be SourceTree. I’m very pleased with it, and if anything my predilection for the Mac platform has increased significantly during this time – I’ve barely used Windows in the last 8 months, and now when I do I find it quite unpleasant. I also went from hating Obj-C (as a C++ user) to really liking it in the space of a few months, and I’m quite happy to include it on my resume now.

The other thing I find, now that I’m juggling fewer balls and am less narrowly focussed, is that my creativity has ramped up considerably. My ‘Project Ideas’ file has swelled significantly during 2010, and I hope to have chance to pick off some of the juiciest of them later in 2011.

So, I guess the important thing in a retrospective is to decide whether I achieved my goals in 2010. Given that my back is much better now, I’m less stressed, more creative and I have a new product out, I guess the answer is yes. Not everything went to plan of course – SourceTree launched later than I’d intended (isn’t it ever thus?) and still has quite a way to go to recoup its investment, my only Mac died a month into me starting Mac-only development (but on the plus side that gave me an excuse to buy a new one, and Apple ended up repairing the other one free even out of warranty), and I’m still not completely recovered health-wise, but all in all, even if it’s not a double rainbow, it’s gotta be at least a rainbow and a half.

My best wishes to everyone for the New Year celebrations, and I hope we’ll all have a happy, healthy and prosperous 2011.

I’m still getting the odd comment on my post in April about my back & why I was retiring from Ogre – thanks again to everyone for the best wishes. I haven’t posted any updates since then, both because I don’t want to ‘count my chickens’ too early, because I’ve been busy, and because I don’t want to be too self-indulgent; but it’s been 6 months now, and I figure some people might like to know my status, because it really has changed a lot.

I’m glad to say it’s good news. In the 6 months since I announced my retirement, my back has improved a huge amount. Withdrawing from my multitude of (over)commitments was hard, and I felt guilty for quite a while (as well as reducing my income since I switched to less stressful projects of my own), but it was absolutely the right decision. Literally every month that passed has made a small difference, and it has thankfully accumulated – right now it’s better than it’s ever been. I’m not in pain on a daily basis any more (only occasionally if I overdo something), and I feel a lot more confident that I’m not going to injure myself doing normal everyday things. You’ve no idea how much it means to me to be able to go to bed at night without dreading how much it’s going to hurt to get out of bed the next day – such a simple thing, but it’s a huge deal to me after 2 years of pain.

My physio’s theory (after being very puzzled because the problems I had were symptomatic of a major trauma, of which there was no evidence) was that the problem was a vicious circle of stress and excessive time being desk-bound that had caused a long-term shortening of the tendons / muscles in my back, coupled with a general age-related degradation of discs, which then (because of the pain) caused me injure the nerves far too easily, causing more stress and making exercise to alleviate the problem long-term very difficult to do. In hindsight, while I was skeptical for a long time it appears he was right – it’s just that to break that cycle required a massive change in lifestyle over a sustained period of time, and being consistent with that change even when it didn’t seem to be working early on.

I’ve always believed that success is just a factor of effort and consistency, and that there’s no such thing as a ‘quick fix’ for anything important. I think if there’s any experience in my life that has reinforced this philosophy (and also tested my ability to follow it), it’s this. Plenty of people (particularly in the USA) told me I should be looking for a surgical solution. I never wanted that – the spine is a ridiculously complicated structure and I had very little faith that such a fix would last (even if it worked short-term), particularly while the original cause of the problem wasn’t explained. I’m glad that the culture here is to operate only if there is no chance of natural rehabilitation, because even if there had there been a viable ‘quick fix’, without the lifestyle change I’m sure the problem would just have reappeared later anyway.

It also seems that there was a combination of physical and psychological factors contributing to this, which again undermines the surgical option. I’ve historically had quite a high tolerance for stress and work-related pressure – in fact I’ve done some of my best work in the crucible of ridiculous deadlines and seemingly intractable problems. My general attitude that I could ‘push through’ any difficulties, reinforced by the inevitable high in the aftermath, pretty much set me up for this kind of problem – because when my health started to falter, I took the same attitude. Obviously, it doesn’t work. Stress and frustration cause tense muscles, which exacerbate the problems of discs compressed by too much inactivity and make stretching them out to their proper state again even more difficult. So in addition to changing my work habits, I’ve had to learn how to relax again. It sounds ridiculous, but because of stress and a reaction to pain (which is to tense up), I had literally forgotten how to relax a certain bunch of muscles in my lower / mid back. I had to be taught how to do it again using breathing exercises and gradual, millimetre-by-millimetre changes in posture over time (too much at once would cause me lots of pain).

So, it’s been a long road and it’s not finished yet – nor will it ever really be finished, since I’ll need to continue with my new lifestyle for the foreseeable future. I have a newfound appreciation for keeping my stress levels lower and being more realistic about what I put my body through in the pursuit of work / hobbies. In many ways the old Steve is gone (the workaholic, coding at 3am Steve who created Ogre). The new Steve has a working back though, and frankly, that’s far more valuable.

Wow, talk about a ‘bolt from the blue’ here: I woke up this morning to find that I’d been included on the ‘Game Developer 50′ in the November 2010 issue of the long established Game Developer Magazine. It’s entirely, completely bonkers to see my name included in the same list as the likes of Sid Meier and Gabe Newell. Just, wow.

Obviously my inclusion was based on my 10 years working on Ogre – it’s somewhat ironic that I was a GDMag subscriber when I started Ogre originally, and did so with the intention of creating my own games with it, inspired by what I read in those pages. Of course, I never really got around to finishing a whole game myself, but playing a part in other people’s games was in many ways just as rewarding. It’s a really, really nice surprise to get included in this list personally – Ogre’s been mentioned a few times in the magazine over the years of course, but to be on this list is the icing on the cake.

I’m also really glad to see Max Schaefer from Runic on the list, the story of Torchlight’s success is now legendary!

So, I’m not sure who was responsible for nominating me on to this list, but you have my sincere thanks. The photo they used was the one of me receiving the Ogre statuette, and the stupid grin I have right now is comparable.

In a complete and total surprise, my cousin presented to me yesterday the result of a grand conspiracy in the Ogre community to commemorate my time as project lead – a specially designed, unique Ogre statuette! Thumbnails below, click for more detail…

I literally had no idea this was going on, or that my cousin had been asked to make the delivery that day (I thought we were just meeting for a social). I was completely taken aback and very touched that the people in the community would go to the trouble of doing something like this for me. There’s a thread in the forums going into more detail on how they organised it, including the many, many different designs they mocked up and considered before voting for this one. I’m humbled and flattered.

My sincere thanks to everyone who was involved in organising this, and very well done for creating such an excellent statue and managing so successfully to keep me in the dark about it! You guys are just awesome.

Yesterday saw a triple-whammy of sugary Apple gaming goodness:

1. Steam for Mac was released, meaning all the games you own on Steam that are ported to the Mac can also be played there, free.
2. Torchlight was a day-1 release on the service, meaning Ogre (and therefore code written by me) was among the very first on the service.
3. Portal became free (for Mac and PC)

Wow. A great day for Mac gaming. I noticed that World Of Goo was up on day 1 too, and since I’d bought it on Steam I could play it right away on my MBP too. Yummy.

Curiously, considering it’s based on Ogre, I don’t actually own Torchlight on Steam – I had a free Windows-only copy from Runic, I bought a physical (Windows-only) copy for my shelf, and I bought copies for both my wife and Diablo-obsessed brother in law on Steam but I never got a copy there myself, so I haven’t tried it on the Mac yet. I need to ask my wife to log in on the MBP so I can try it!

For 18 months I’ve been told by a succession of doctors and physios that I didn’t have anything structurally wrong with my spine and that my bouts of back pain were simply ‘standard non-specific back pain’ – ie muscle problems that I should just take NSAIDs for and exercise more. I’d been a bit skeptical because the problems were occasionally quite extreme and seemed to be always centred on one particular location (the joint just at the bottom of my ribcage), but after getting many opinions and one set of x-rays I went along with it.

Things have been quite good recently, up to mid-February when I had a bit of a relapse for a few weeks after doing a little too much. I raised it with my doctor again, explained that I’d been doing all the exercise and going to the gym as recommended, and yet it still flared up at what I considered to be fairly minor provocation. He scheduled me in for another set of x-rays which I expected to not come back with anything conclusive since the last set didn’t (and you can’t get into the MRI scan here unless you go through this step again first, allegedly). They took more pictures this time but I didn’t expect much given all the opinions so far.

Imagine my surprise therefore that when I got the results today, they actually had a concrete explanation for me. Apparently in my lower thoracic (ie exactly where I’d been pointing all these months) I have some disc degeneration and calcification going on, which is what is causing the stiffness and pain. This is something that happens with age anyway, but given my relative(!) youth (36) they thought it looked like it might be a result of either a trauma such as a sports injury – I can’t think of anything – or sometimes they see it in people who were child gymnasts – again not something I can attest to! Basically, something has happened to make my spine degenerate in that area faster than it should have done for my age. Too many hours spent stressed out at a desk may have been a contributing factor in that, although he thought it would have to be a lot of hours and probably combined with other factors.

So anyway, the ‘good’ news is that I actually have a reason now, an explanation for why I’m so susceptible to strains and stress on my back these days. In a way it’s nice to have something to point at. The bad news is that this isn’t fixable, it can merely be managed via careful exercise and lifestyle changes – many of which I’ve made already but I probably need to go even further. The prognosis is that I should be able to live pain-free so long as I manage it carefully over the long term to stop it degenerating further.

Following this analysis, I’ve been prompted to make a decision which I’ve been reluctantly considering for a while anyway – I’m retiring as OGRE Project Lead. I’ve thoroughly enjoyed my 10 years leading OGRE from unknown personal project to where we are today, but leading an open source project requires an enormous amount of dedication, passion, and above all an awful lot of time spent at a keyboard, most often in addition to a ‘regular job’ with which to pay the bills, and I feel I just can’t give that to the level that’s required any more. It will be with no small amount of sadness that I finally take off the leader’s hat – which by now is quite battered and worn in.

I still intend to be around and involved in the project – I’ll be contributing some code, giving advice when it’s wanted, and overseeing the establishment of an OGRE Foundation to handle the donations and funding side, but the days of me living and breathing OGRE, vetting every change, and being the person with whom the buck stops when there’s a bug, will be over. I’ll basically be contributing what and when I can, but shrugging off the responsibility and expectation that is inevitably associated with being the lead developer.

We have a great team and community around OGRE and I’m sure the project will be fine with me taking a more back-seat role – time for younger and less physically challenged developers to step into the limelight

I’ve had requests to post the OS & browser stats for the OGRE site, which I didn’t include in my previous demographics post, so here we go.

A caveat to start off with – as a programmer-oriented site our users are obviously a little different in their choice of tech to the population at large!

Operating Systems

Not really a surprise there, Windows dominates the landscape, with Linux and OS X pulling up the rear. Personally on the desktop I’m a Windows and OS X user so my visits are contributing to those rows. There’s a decent showing for the mobile platforms too, iPhone and iPod particularly, a fair few on Android and even some early iPad hits.

Windows Breakdown

It’s worth drilling down into the huge Windows stat to see what versions people are using:

XP still rules the roost then, and thank goodness Vista is sinking rapidly to the bottom like the cast-iron turd sculpture it is – Windows 7 has already more than doubled its share. And it’s amusing to see that a couple of crazy b?st#rds are still running ME and NT.

Mac OS X Breakdown

I’d heard that supporting pre-Leopard versions of Mac OS X was increasingly becoming pointless since almost everyone had upgraded, and these stats bear that out – 96% of users are running 10.5 (Leopard) or better.

Even more interesting was the sole visitor running 10.7! Was that an error, or did we have someone from Apple on an unreleased future version visiting the site? And what’s that ’68K’ entry about – someone running an Apple ][ emulator or something?

The Linux breakdown wasn’t interesting (99% ‘not set’, the rest just kernel versions), so there’s nothing to post for that – don’t email me about  missing out Linux please

Browsers

Firefox is the clear winner here at a huge 50% – this definitely reflects our developer-focussed audience. In fact, I used to be an avid Firefox user until quite recently, when the new version of Chrome added the equivalent to the extensions I relied on in Firefox, at which point I switched because Chrome is more efficient with memory in particular. IE’s share at 18% is definitely not representative of the general user population, but then our users tend to be a bit better informed than that

So there you go, request fulfilled – hope it was interesting.

Ok, so a new clause in the Terms of Service for Apple’s newly announced iPhone OS 4 is understandably causing some consternation around the internet:

“3.3.1 … Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).”

The common understanding is that this is a shot across Adobe’s bow, but also aimed at people creating emulation environments. Potential justifications for this could include performance concerns (given the new multitasking feature in 4), wanting to avoid shovelware ports from other platforms with no iPhone-specific features to make their platform stand out, or sheer bloody mindedness and wish to tie developers directly to their APIs to minimise the chance that they’ll deploy on competitors machines.

As a general principle, I don’t like this sort of thing – telling developers what they can and can’t do is stifling. But, I had to take away one positive from it – a company telling people to use native code, instead of the opposite which I’ve seen too much of lately. In recent years, the likes of Microsoft have insisted that developers use their intermediate VM layers to deploy on some devices (XNA, Windows Phone 7) – regardless that these environments have about 20 years less maturity (in terms of libraries and existing code) than what I already have in C & C++. Having them tell me that no, despite all these great battle-tested libraries that I’m used to using, instead I have to use comparatively immature ports and replacements of varying quality, just because they tell me so. That drives me nuts – sure, let’s throw away and re-invent hundreds of functioning & tested libraries just because…well, just because! They’re old and we’re new and awesome! Hmm.

So while Apple telling developers what they can and can’t use is still very wrong from a point of principle, I’m actually glad that someone is championing native code for once, rather than pushing a VM environment. I’d prefer they didn’t mandate anything at all, but I can’t deny a certain urge to fist-bump when native code was the one to get the seal of approval, after getting the impression from other companies that they’d rather no-one had access to the underlying workings of the machine. I like native code, there’s a certain purity about it – and maybe it’s like a sad old gear-head going on about how great the old V8′s used to be, but I don’t care Mostly it’s about my frustration with being forced to discard perfectly good native libraries and look for / build replacements for no good reason.

PS For the record, OGRE on iPhone isn’t affected by this new ToS because we’re 100% native, baby.

[edit]For those pointing out that C# and such eventually run on native code anyway – that’s not the point. The point is that on certain devices – XNA and Windows Phone 7 – you simply cannot use libraries that were not written in .Net originally, meaning that years worth of dev libraries are inaccessible and need to be (pointlessly IMO) rewritten in .Net. And yes, this is exactly the same as Apple are doing here (but in reverse), if you interpret it in the strictest sense that you’re only allowed to use code written in Obj-C, C and C++. I’m just taking a perverse delight in the fact that it’s C/C++ libraries from the last 20+ years that are on the winning side for a change, instead of being the ones that are excluded (which frankly I’m completely sick of).

Almost exactly three years ago, I posted an analysis of the traffic on ogre3d.org and the rough country breakdown of our users, which is always fascinating to me. I hadn’t actually been collecting web stats on the site for about a year (the previous set-up was lost when I had to recreate the server in a hurry, and somehow reinstating it never seemed to rise to the top of my TODO list), but a month ago I finally got around to adding Google Analytics to the site. The results have been very interesting, particularly when compared to 3 years ago, so I thought I’d share some factoids with you.

Visitors still increasing

In 2007, log analysis indicated we were getting a little over 100,000 unique visitors per month; obviously this is not 100% accurate due to shared & dynamic IP addresses, people logging in from multiple sites, etc, but it’s a reasonable order of magnitude indicator. Analytics is typically more conservative in its figures, since it excludes bots better as well as non-Javascript browsers, but still in the past month (actually only 28 days) we’ve had over 120,000 unique visitors to the site – and 1.2M page views – so we’ve sustained and slightly improved our user traffic. And all without any Slashdot posts Also, this doesn’t count visits to Sourceforge, BitBucket or ome static generated HTML like our online documentation pages.

The Meteoric Rise of China

In 2007, China ranked 15th in our league table of visitors. Three years on and they’ve risen to the number two spot, comfortably surpassing Germany – at first I wondered if that was down to users there using fewer proxies, but since the figures for other countries have remained fairly stable I think the majority of this is genuinely a vast increase in the number of Chinese visitors to OGRE’s site. Here’s the top 10 countries (figures are for the number of visits):

The range of countries is demonstrated by how many are in the grey ‘others’ section (38.73%). Except for the massive change in China’s share, most of the other countries have stayed approximately in their relative positions & shares of the user population since 2007.

Region View – Europe still dominates, Asia challenges the Americas

The country view is, however, quite misleading if your aim is to decide where to locate a web server for example, because it naturally biases the figures towards large unified countries (like the USA and China), and doesn’t really show a true regional picture. For that, we have to examine the numbers (again, number of visits) by continent:

Now, even though the continent view includes Russia in Europe which screws up the locality principle a bit, even if you exclude that Europe dominates our community, with close to 1 in every 2 visitors to the site being from Europe. The Americas  and Asia share most of the rest almost equally now, which is a change from 2007 when the Americas were more dominant, and everyone else shares the scraps (3.5%). The Americas figure is made up of about 86% North America and 14% South America, and Asia is predominantly (60%) the Eastern Asia countries (mostly China, but South Korea holds its own too), with South-East and Southern Asia sharing the rest – particular hotspots there are India, Indonesia, Malasia, Turkey and Vietnam.

City Clusters

One thing I like about Analytics is the ability to drill down into countries and look at the local clustering. There are the expected clusters around cities – in the USA, the top 2 cities are unsurprisingly New York and Los Angeles, although Columbus OH takes the number 3 spot, and in the UK the clustering around London is massive – but they typically represent only about 25-30% of the audience, with the rest being scattered pretty much uniformly across most areas of the country in question. It’s fun to be able to point at almost anywhere in Europe, North America and the southern and eastern parts of Asia and to have a pretty good chance of being quite near to someone who has used the OGRE site.

The Monday morning OGRE fix

With OGRE obviously used by a lot of people in their spare time, you might expect that the weekends would be the busiest times for the site, but the opposite is in fact true – Mondays are consistently the busiest days (particularly 6-9am PST), with Saturday being the least busy. Whether this is because people are working with OGRE, or just cheekily surfing in their work time rather than face the Monday workload, is hard to verify!

Final Thought

I get a kick out of looking at these stats so I hope you find them interesting too. It’s really cool to think that there are only a very small number of countries (such as North Korea and Laos) from which we don’t get any (non-proxy) visitors from in a typical month, and it’s very interesting to see how the visitor base is gradually spreading out and diversifying, something which I’m sure every site witnesses but it’s interesting to see it in your own data. The question is – will China keep the current trajectory? At this rate, they’ll take the number one spot from the USA in only a few more years and put Asia second in the regional rankings!

Writing good documentation is hard. While I happen to think that API references generated from source code can be extremely useful, they’re only part of the story, and eventually everyone needs to write something more substantial for their software. You can get away with writing HTML directly, and separately using a word processor to write PDFs for so long, but eventually you need a proper tool chain with the following characteristics:

• Lets the author concentrate on content rather than style
• Generates multiple formats from one source (HTML, PDF, man pages, HTML Help etc)
• Does all the tedious work for you such as TOCs, cross-references, source code highlighting, footnotes
• Is friendly to source control systems & diffs in general
• Standard enough that you could submit the content to a publisher if you wanted to
• Preferably cross-platform, standards-based and not oriented to any particular language or technology

When I came to write the OGRE manual many, many years ago, I went with Texinfo – it seemed a good idea at the time, and ticked most of the boxes above. The syntax is often a bit esoteric, and the tools used to generate output frequently a bit flaky (texi2html has caused me many headaches over the years thanks to  poorly documented breaking changes), but it worked most of the time.

I’ve been meaning to replace this tool chain with something else for new projects for a while, and DocBook sprung to mind since it’s the ‘new standard’ for technical documentation. It’s quite popular with open source projects now and it’s the preferred format for many publishers such as O’Reilly. In the short term, I want to write some developer instructions for OGRE for our future Mercurial setup, but in the long term, I’d really like a good documentation tool chain for all sorts of other purposes, and Texinfo feels increasingly unsatisfactory these days.

Having spent some time this week establishing a new working tool chain, and encountering & resolving a number of issues along the way, I thought I’d share my setup with you.

Read the rest of this entry »