So, 2010. I started this year with three significant and intersecting goals: to significantly simplify my working life, to reduce my stress levels, and to spend more time in the company of my own creativity by working on my own projects. The reason was this: by the end of 2009 several factors had led me to a situation where I had my fingers in too many pies, too many balls in the air, too many – well, you get the picture. The thing with being a consultant / contractor is that work comes in with a very uncertain frequency; projects have a tendency to ‘bunch up’, and because of the inevitable lean periods in between you don’t really want to say ‘no’ to anything. The economic situation since late 2008 also led to projects being noticeably more cost-sensitive, meaning more juggling required to fill out a coherent schedule than it had been before that. All this could get quite stressful, and with a back problem to nurse and an open source project to run on top, things weren’t that much fun at times. So my resolution for 2010 was to stop just trying to make the status quo work, and instead to do things my way – to ditch smaller pieces of work, to only get involved in larger projects that I found personally interesting, and to spend the rest of the time investing in my own projects instead. Fundamentally it was about taking control back.

At that time I didn’t plan to retire from OGRE, but in hindsight I think subconsciously I’d made the decision already, I just hadn’t admitted it to myself yet. I’m extremely proud of what I managed to accomplish with OGRE with the help of countless contributors, and was sad to put that role aside after 10 years, but in many ways it had become a rod for my own back. Not only was it a hugely time consuming, 24/7 job to look after, but there was an inherent assumption, from others and from myself, that projects I chose to work on would naturally be associated with OGRE in some way. That was quite a pair of blinkers to have on, even if I didn’t know I was wearing them. If success with a product or in a subject area has a downside, it’s probably that it can become a gravity well which resists you exploring other areas of interest. So while I’m sad to have retired from OGRE and look back on my time on the project with a great deal of fondness and pride, I can’t deny that I don’t regret casting off and returning to the open seas, even if the security of the harbour was comforting. Ok, I’m done with the sailing metaphors now.

One unexplored continent (damn!) which was calling me siren-like (ok, I really am done now) was native Mac development. I bought my first Mac in 2007, initially only to support OGRE on it (before masterfalcon came along with his superlative Apple-fu), but I’ve since been converted to a fully card-carrying Mac nutcase. So, I decided to finally go all-in and learn Objective-C and Cocoa, and make my own Mac-only tool, which of course turned out to be SourceTree. I’m very pleased with it, and if anything my predilection for the Mac platform has increased significantly during this time – I’ve barely used Windows in the last 8 months, and now when I do I find it quite unpleasant. I also went from hating Obj-C (as a C++ user) to really liking it in the space of a few months, and I’m quite happy to include it on my resume now.

The other thing I find, now that I’m juggling fewer balls and am less narrowly focussed, is that my creativity has ramped up considerably. My ‘Project Ideas’ file has swelled significantly during 2010, and I hope to have chance to pick off some of the juiciest of them later in 2011.

So, I guess the important thing in a retrospective is to decide whether I achieved my goals in 2010. Given that my back is much better now, I’m less stressed, more creative and I have a new product out, I guess the answer is yes. Not everything went to plan of course – SourceTree launched later than I’d intended (isn’t it ever thus?) and still has quite a way to go to recoup its investment, my only Mac died a month into me starting Mac-only development (but on the plus side that gave me an excuse to buy a new one, and Apple ended up repairing the other one free even out of warranty), and I’m still not completely recovered health-wise, but all in all, even if it’s not a double rainbow, it’s gotta be at least a rainbow and a half.

My best wishes to everyone for the New Year celebrations, and I hope we’ll all have a happy, healthy and prosperous 2011.

I’m glad to say it’s good news. In the 6 months since I announced my retirement, my back has improved a huge amount. Withdrawing from my multitude of (over)commitments was hard, and I felt guilty for quite a while (as well as reducing my income since I switched to less stressful projects of my own), but it was absolutely the right decision. Literally every month that passed has made a small difference, and it has thankfully accumulated – right now it’s better than it’s ever been. I’m not in pain on a daily basis any more (only occasionally if I overdo something), and I feel a lot more confident that I’m not going to injure myself doing normal everyday things. You’ve no idea how much it means to me to be able to go to bed at night without dreading how much it’s going to hurt to get out of bed the next day – such a simple thing, but it’s a huge deal to me after 2 years of pain.

My physio’s theory (after being very puzzled because the problems I had were symptomatic of a major trauma, of which there was no evidence) was that the problem was a vicious circle of stress and excessive time being desk-bound that had caused a long-term shortening of the tendons / muscles in my back, coupled with a general age-related degradation of discs, which then (because of the pain) caused me injure the nerves far too easily, causing more stress and making exercise to alleviate the problem long-term very difficult to do. In hindsight, while I was skeptical for a long time it appears he was right – it’s just that to break that cycle required a massive change in lifestyle over a sustained period of time, and being consistent with that change even when it didn’t seem to be working early on.

I’ve always believed that success is just a factor of effort and consistency, and that there’s no such thing as a ‘quick fix’ for anything important. I think if there’s any experience in my life that has reinforced this philosophy (and also tested my ability to follow it), it’s this. Plenty of people (particularly in the USA) told me I should be looking for a surgical solution. I never wanted that – the spine is a ridiculously complicated structure and I had very little faith that such a fix would last (even if it worked short-term), particularly while the original cause of the problem wasn’t explained. I’m glad that the culture here is to operate only if there is no chance of natural rehabilitation, because even if there had there been a viable ‘quick fix’, without the lifestyle change I’m sure the problem would just have reappeared later anyway.

It also seems that there was a combination of physical and psychological factors contributing to this, which again undermines the surgical option. I’ve historically had quite a high tolerance for stress and work-related pressure – in fact I’ve done some of my best work in the crucible of ridiculous deadlines and seemingly intractable problems. My general attitude that I could ‘push through’ any difficulties, reinforced by the inevitable high in the aftermath, pretty much set me up for this kind of problem – because when my health started to falter, I took the same attitude. Obviously, it doesn’t work. Stress and frustration cause tense muscles, which exacerbate the problems of discs compressed by too much inactivity and make stretching them out to their proper state again even more difficult. So in addition to changing my work habits, I’ve had to learn how to relax again. It sounds ridiculous, but because of stress and a reaction to pain (which is to tense up), I had literally forgotten how to relax a certain bunch of muscles in my lower / mid back. I had to be taught how to do it again using breathing exercises and gradual, millimetre-by-millimetre changes in posture over time (too much at once would cause me lots of pain).

So, it’s been a long road and it’s not finished yet – nor will it ever really be finished, since I’ll need to continue with my new lifestyle for the foreseeable future. I have a newfound appreciation for keeping my stress levels lower and being more realistic about what I put my body through in the pursuit of work / hobbies. In many ways the old Steve is gone (the workaholic, coding at 3am Steve who created Ogre). The new Steve has a working back though, and frankly, that’s far more valuable.

Obviously my inclusion was based on my 10 years working on Ogre – it’s somewhat ironic that I was a GDMag subscriber when I started Ogre originally, and did so with the intention of creating my own games with it, inspired by what I read in those pages. Of course, I never really got around to finishing a whole game myself, but playing a part in other people’s games was in many ways just as rewarding. It’s a really, really nice surprise to get included in this list personally – Ogre’s been mentioned a few times in the magazine over the years of course, but to be on this list is the icing on the cake.

I’m also really glad to see Max Schaefer from Runic on the list, the story of Torchlight’s success is now legendary!

So, I’m not sure who was responsible for nominating me on to this list, but you have my sincere thanks. The photo they used was the one of me receiving the Ogre statuette, and the stupid grin I have right now is comparable.

I literally had no idea this was going on, or that my cousin had been asked to make the delivery that day (I thought we were just meeting for a social). I was completely taken aback and very touched that the people in the community would go to the trouble of doing something like this for me. There’s a thread in the forums going into more detail on how they organised it, including the many, many different designs they mocked up and considered before voting for this one. I’m humbled and flattered.

My sincere thanks to everyone who was involved in organising this, and very well done for creating such an excellent statue and managing so successfully to keep me in the dark about it! You guys are just awesome.

1. Steam for Mac was released, meaning all the games you own on Steam that are ported to the Mac can also be played there, free.
2. Torchlight was a day-1 release on the service, meaning Ogre (and therefore code written by me) was among the very first on the service.
3. Portal became free (for Mac and PC)

Wow. A great day for Mac gaming. I noticed that World Of Goo was up on day 1 too, and since I’d bought it on Steam I could play it right away on my MBP too. Yummy.

Curiously, considering it’s based on Ogre, I don’t actually own Torchlight on Steam – I had a free Windows-only copy from Runic, I bought a physical (Windows-only) copy for my shelf, and I bought copies for both my wife and Diablo-obsessed brother in law on Steam but I never got a copy there myself, so I haven’t tried it on the Mac yet. I need to ask my wife to log in on the MBP so I can try it!

Things have been quite good recently, up to mid-February when I had a bit of a relapse for a few weeks after doing a little too much. I raised it with my doctor again, explained that I’d been doing all the exercise and going to the gym as recommended, and yet it still flared up at what I considered to be fairly minor provocation. He scheduled me in for another set of x-rays which I expected to not come back with anything conclusive since the last set didn’t (and you can’t get into the MRI scan here unless you go through this step again first, allegedly). They took more pictures this time but I didn’t expect much given all the opinions so far.

Imagine my surprise therefore that when I got the results today, they actually had a concrete explanation for me. Apparently in my lower thoracic (ie exactly where I’d been pointing all these months) I have some disc degeneration and calcification going on, which is what is causing the stiffness and pain. This is something that happens with age anyway, but given my relative(!) youth (36) they thought it looked like it might be a result of either a trauma such as a sports injury – I can’t think of anything – or sometimes they see it in people who were child gymnasts – again not something I can attest to! Basically, something has happened to make my spine degenerate in that area faster than it should have done for my age. Too many hours spent stressed out at a desk may have been a contributing factor in that, although he thought it would have to be a lot of hours and probably combined with other factors.

So anyway, the ‘good’ news is that I actually have a reason now, an explanation for why I’m so susceptible to strains and stress on my back these days. In a way it’s nice to have something to point at. The bad news is that this isn’t fixable, it can merely be managed via careful exercise and lifestyle changes – many of which I’ve made already but I probably need to go even further. The prognosis is that I should be able to live pain-free so long as I manage it carefully over the long term to stop it degenerating further.

Following this analysis, I’ve been prompted to make a decision which I’ve been reluctantly considering for a while anyway – I’m retiring as OGRE Project Lead. I’ve thoroughly enjoyed my 10 years leading OGRE from unknown personal project to where we are today, but leading an open source project requires an enormous amount of dedication, passion, and above all an awful lot of time spent at a keyboard, most often in addition to a ‘regular job’ with which to pay the bills, and I feel I just can’t give that to the level that’s required any more. It will be with no small amount of sadness that I finally take off the leader’s hat – which by now is quite battered and worn in.

I still intend to be around and involved in the project – I’ll be contributing some code, giving advice when it’s wanted, and overseeing the establishment of an OGRE Foundation to handle the donations and funding side, but the days of me living and breathing OGRE, vetting every change, and being the person with whom the buck stops when there’s a bug, will be over. I’ll basically be contributing what and when I can, but shrugging off the responsibility and expectation that is inevitably associated with being the lead developer.

We have a great team and community around OGRE and I’m sure the project will be fine with me taking a more back-seat role – time for younger and less physically challenged developers to step into the limelight

A caveat to start off with – as a programmer-oriented site our users are obviously a little different in their choice of tech to the population at large!

Operating Systems

Not really a surprise there, Windows dominates the landscape, with Linux and OS X pulling up the rear. Personally on the desktop I’m a Windows and OS X user so my visits are contributing to those rows. There’s a decent showing for the mobile platforms too, iPhone and iPod particularly, a fair few on Android and even some early iPad hits.

Windows Breakdown

It’s worth drilling down into the huge Windows stat to see what versions people are using:

XP still rules the roost then, and thank goodness Vista is sinking rapidly to the bottom like the cast-iron turd sculpture it is – Windows 7 has already more than doubled its share. And it’s amusing to see that a couple of crazy b?st#rds are still running ME and NT.

Mac OS X Breakdown

I’d heard that supporting pre-Leopard versions of Mac OS X was increasingly becoming pointless since almost everyone had upgraded, and these stats bear that out – 96% of users are running 10.5 (Leopard) or better.

Even more interesting was the sole visitor running 10.7! Was that an error, or did we have someone from Apple on an unreleased future version visiting the site? And what’s that ’68K’ entry about – someone running an Apple ][ emulator or something?

The Linux breakdown wasn’t interesting (99% ‘not set’, the rest just kernel versions), so there’s nothing to post for that – don’t email me about  missing out Linux please

Browsers

Firefox is the clear winner here at a huge 50% – this definitely reflects our developer-focussed audience. In fact, I used to be an avid Firefox user until quite recently, when the new version of Chrome added the equivalent to the extensions I relied on in Firefox, at which point I switched because Chrome is more efficient with memory in particular. IE’s share at 18% is definitely not representative of the general user population, but then our users tend to be a bit better informed than that

So there you go, request fulfilled – hope it was interesting.

“3.3.1 … Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).”

The common understanding is that this is a shot across Adobe’s bow, but also aimed at people creating emulation environments. Potential justifications for this could include performance concerns (given the new multitasking feature in 4), wanting to avoid shovelware ports from other platforms with no iPhone-specific features to make their platform stand out, or sheer bloody mindedness and wish to tie developers directly to their APIs to minimise the chance that they’ll deploy on competitors machines.

As a general principle, I don’t like this sort of thing – telling developers what they can and can’t do is stifling. But, I had to take away one positive from it – a company telling people to use native code, instead of the opposite which I’ve seen too much of lately. In recent years, the likes of Microsoft have insisted that developers use their intermediate VM layers to deploy on some devices (XNA, Windows Phone 7) – regardless that these environments have about 20 years less maturity (in terms of libraries and existing code) than what I already have in C & C++. Having them tell me that no, despite all these great battle-tested libraries that I’m used to using, instead I have to use comparatively immature ports and replacements of varying quality, just because they tell me so. That drives me nuts – sure, let’s throw away and re-invent hundreds of functioning & tested libraries just because…well, just because! They’re old and we’re new and awesome! Hmm.

So while Apple telling developers what they can and can’t use is still very wrong from a point of principle, I’m actually glad that someone is championing native code for once, rather than pushing a VM environment. I’d prefer they didn’t mandate anything at all, but I can’t deny a certain urge to fist-bump when native code was the one to get the seal of approval, after getting the impression from other companies that they’d rather no-one had access to the underlying workings of the machine. I like native code, there’s a certain purity about it – and maybe it’s like a sad old gear-head going on about how great the old V8′s used to be, but I don’t care Mostly it’s about my frustration with being forced to discard perfectly good native libraries and look for / build replacements for no good reason.

PS For the record, OGRE on iPhone isn’t affected by this new ToS because we’re 100% native, baby.

[edit]For those pointing out that C# and such eventually run on native code anyway – that’s not the point. The point is that on certain devices – XNA and Windows Phone 7 – you simply cannot use libraries that were not written in .Net originally, meaning that years worth of dev libraries are inaccessible and need to be (pointlessly IMO) rewritten in .Net. And yes, this is exactly the same as Apple are doing here (but in reverse), if you interpret it in the strictest sense that you’re only allowed to use code written in Obj-C, C and C++. I’m just taking a perverse delight in the fact that it’s C/C++ libraries from the last 20+ years that are on the winning side for a change, instead of being the ones that are excluded (which frankly I’m completely sick of).

Visitors still increasing

In 2007, log analysis indicated we were getting a little over 100,000 unique visitors per month; obviously this is not 100% accurate due to shared & dynamic IP addresses, people logging in from multiple sites, etc, but it’s a reasonable order of magnitude indicator. Analytics is typically more conservative in its figures, since it excludes bots better as well as non-Javascript browsers, but still in the past month (actually only 28 days) we’ve had over 120,000 unique visitors to the site – and 1.2M page views – so we’ve sustained and slightly improved our user traffic. And all without any Slashdot posts Also, this doesn’t count visits to Sourceforge, BitBucket or ome static generated HTML like our online documentation pages.

The Meteoric Rise of China

In 2007, China ranked 15th in our league table of visitors. Three years on and they’ve risen to the number two spot, comfortably surpassing Germany – at first I wondered if that was down to users there using fewer proxies, but since the figures for other countries have remained fairly stable I think the majority of this is genuinely a vast increase in the number of Chinese visitors to OGRE’s site. Here’s the top 10 countries (figures are for the number of visits):

The range of countries is demonstrated by how many are in the grey ‘others’ section (38.73%). Except for the massive change in China’s share, most of the other countries have stayed approximately in their relative positions & shares of the user population since 2007.

Region View – Europe still dominates, Asia challenges the Americas

The country view is, however, quite misleading if your aim is to decide where to locate a web server for example, because it naturally biases the figures towards large unified countries (like the USA and China), and doesn’t really show a true regional picture. For that, we have to examine the numbers (again, number of visits) by continent:

Now, even though the continent view includes Russia in Europe which screws up the locality principle a bit, even if you exclude that Europe dominates our community, with close to 1 in every 2 visitors to the site being from Europe. The Americas  and Asia share most of the rest almost equally now, which is a change from 2007 when the Americas were more dominant, and everyone else shares the scraps (3.5%). The Americas figure is made up of about 86% North America and 14% South America, and Asia is predominantly (60%) the Eastern Asia countries (mostly China, but South Korea holds its own too), with South-East and Southern Asia sharing the rest – particular hotspots there are India, Indonesia, Malasia, Turkey and Vietnam.

City Clusters

One thing I like about Analytics is the ability to drill down into countries and look at the local clustering. There are the expected clusters around cities – in the USA, the top 2 cities are unsurprisingly New York and Los Angeles, although Columbus OH takes the number 3 spot, and in the UK the clustering around London is massive – but they typically represent only about 25-30% of the audience, with the rest being scattered pretty much uniformly across most areas of the country in question. It’s fun to be able to point at almost anywhere in Europe, North America and the southern and eastern parts of Asia and to have a pretty good chance of being quite near to someone who has used the OGRE site.

The Monday morning OGRE fix

With OGRE obviously used by a lot of people in their spare time, you might expect that the weekends would be the busiest times for the site, but the opposite is in fact true – Mondays are consistently the busiest days (particularly 6-9am PST), with Saturday being the least busy. Whether this is because people are working with OGRE, or just cheekily surfing in their work time rather than face the Monday workload, is hard to verify!

Final Thought

I get a kick out of looking at these stats so I hope you find them interesting too. It’s really cool to think that there are only a very small number of countries (such as North Korea and Laos) from which we don’t get any (non-proxy) visitors from in a typical month, and it’s very interesting to see how the visitor base is gradually spreading out and diversifying, something which I’m sure every site witnesses but it’s interesting to see it in your own data. The question is – will China keep the current trajectory? At this rate, they’ll take the number one spot from the USA in only a few more years and put Asia second in the regional rankings!

• Lets the author concentrate on content rather than style
• Generates multiple formats from one source (HTML, PDF, man pages, HTML Help etc)
• Does all the tedious work for you such as TOCs, cross-references, source code highlighting, footnotes
• Is friendly to source control systems & diffs in general
• Standard enough that you could submit the content to a publisher if you wanted to
• Preferably cross-platform, standards-based and not oriented to any particular language or technology

When I came to write the OGRE manual many, many years ago, I went with Texinfo – it seemed a good idea at the time, and ticked most of the boxes above. The syntax is often a bit esoteric, and the tools used to generate output frequently a bit flaky (texi2html has caused me many headaches over the years thanks to  poorly documented breaking changes), but it worked most of the time.

I’ve been meaning to replace this tool chain with something else for new projects for a while, and DocBook sprung to mind since it’s the ‘new standard’ for technical documentation. It’s quite popular with open source projects now and it’s the preferred format for many publishers such as O’Reilly. In the short term, I want to write some developer instructions for OGRE for our future Mercurial setup, but in the long term, I’d really like a good documentation tool chain for all sorts of other purposes, and Texinfo feels increasingly unsatisfactory these days.

Having spent some time this week establishing a new working tool chain, and encountering & resolving a number of issues along the way, I thought I’d share my setup with you.

Overview

The tool chain I’ve settled on is shown below. I’ve concentrated on the PDF output route because this tends to be the most complex:

I chose to use AsciiDoc, which was kindly suggested to me by tuan kuranes, in order to produce my DocBook XML from plain text. DocBook is XML, and like all XML, editing it by hand is an overly verbose affair, and prone to human error – specifically DocBook has a lot of long tag names which makes typing them very, very tedious and generally inefficient. Now, of course you can use GUI tools to hide these inefficiencies from you, and to that end you might want to try tools like XMLMind (free for non-commercial use) and Syntext (open source edition) to do this, or even just Eclipse or Visual Studio with the DTD hooked in for auto-completion. But there are 2 reasons why I chose not to do that:

1. Maybe I’m just old-skool, but to me relying on a GUI tool to mask how awkward content is to write is the coding equivalent of ‘papering over the cracks’. I would rather resolve the fundamental problem than simply hide it from myself.
2. Even if you hide the XML from your own eyes, it causes problems elsewhere. XML doesn’t diff very well, and XML generated from a GUI tool diffs even less well, making source control of the content less useful; change tracking is cumbersome and merging an absolute nightmare.

AsciiDoc plain text content is fast and simple to write, it’s easily readable, diffs well and doesn’t require any special editing tools. It’s similar to Texinfo, but less esoteric and the tools are far more modern.

Customising the tool chain

AsciiDoc comes with the ability to generate HTML (single or chunked) or DocBook XML directly, and you can feed the DocBook through other transformations to produce PDF or HTML Help files. I was particularly interested in the PDF route, for which AsciiDoc provides an automated toolchain called a2x. However, I found that this wasn’t good enough for my needs, because the default XSL process wasn’t capable of handling code syntax highlighting & line numbers, unless you used dblatex rather than FOP as the PDF generator which adds its own problems. dblatex is awkward to get working on Windows, and it also requires a LaTeX implementation – a route which seemed perverse to me since I was trying to avoid ‘old’ tools. If I was going to end up going through LaTeX I might as well just write it in that in the first place! So it seemed a hack to go that way, I wanted to keep the direct DocBook to PDF generation.

So, the answer was to replace a2x with my own series of steps, which replaced the XSL processing (which is done with xsltproc in a2x) with Saxon, a Java-based XSL processor which can be extended with syntax highlighting relatively easily.

Installing the tools

I’ll go over the installation on Windows, but it’s mostly the same on Mac / Linux, if anything slightly easier since the packages are often already installed. Edit: I’ve successfully followed my own instructions here and got the same tool chain running great on Mac OS X, so Linux should be fine too. /Edit You will need:

• AsciiDoc (I’m using 8.5.3)
• Python (I’ve tested with 2.5 and 2.6)
• Saxon 6.5 (the latest Saxon is 9 but that’s for XSL 2.0, DocBook is XSL 1.0 and Saxon 6.5 is the stable for that)
• Apache FOP (I’m using 0.95)
• Java runtime (I’m using 1.6)
• XSLTHL syntax highlighting implementation
• Technically optional, to avoid network access on processing DocBook, but IMO you want this:
  • DocBook XML and DocBook XSL files
  • Apache Commons resolver.jar

Note: optionally you can also install XML tools such as xmllint to validate the XML you’re producing, but you don’t need to do this generally. AsciiDoc will normally expect you to install xsltproc and xmllint  for example for a2x to work, but we won’t be using it.

Base Configuration

1. Run the installers for Python and Java if you haven’t done already, and make sure they’re both on your system path
2. Extract the other archives somewhere on your disk – personally I put them all underneath a common ‘DocBook’ root folder
3. Add the FOP root folder to your system path too – not necessarily required but it’s convenient

Create local mappings for DocBook URIs

When processing XSL, Saxon will quite happily pull DocBook definitions from the internet, but this will slow the process down, which is especially noticeable when processing small documents. So, instead I favour making the global URIs resolve to the local copies of the DocBook XML and DocBook XSL that we downloaded earlier. In Saxon, you do this via a CatalogManager.properties file which must be somewhere on your Java classpath. For me, the file looked like this:

catalogs=E:/UsefulResources/DocBook/docbook-xsl-1.75.2/catalog.xml;E:/UsefulResources/DocBook/docbook/4.5/catalog.xml
relative-catalogs=false
static-catalog=yes
catalog-class-name=org.apache.xml.resolver.Resolver
verbosity=1

Notice how I just reference the ‘catalog.xml’ files that are included in the DocBook XML and XSL distributions; these files include the mappings from the URIs to the local files. I saved this file as e:\UsefulResources\DocBook\CatalogManager.properties, and added this folder to my classpath (see ‘Running the toolchain’ below). You can skip this step if you want, but it will speed up your XSL processing considerably so it’s highly recommended.

Defining a custom style

AsciiDoc helpfully comes with some default stylesheets for PDF output, but I supplemented them to get source code highlighting and line numbering working via XSLTHL, which doesn’t work (via FOP) in the standard release. Also, I figured that I’m going to want to customise the styles eventually anyway, so I might as well know how to do it. It’s actually very easy – just define an XSL stylesheet which pulls in the existing AsciiDoc base definitions, and adds to them.

Here’s my simple stylesheet example: fo_steve.xsl.

Most of this file is simply referencing the base stylesheets, enabling syntax colouring and line numbering (where opted for), and doing a single simple style tweak to code syntax highlighting – setting keywords to be red. Obviously there’s lots more you can do, this was just a test to prove the concept works.

Running the tool chain

It might look like there are a lot of moving parts here, but it actually fits together fairly simply, especially once you’ve done the setup as described above. Processing a text file to PDF is simply 3 stages, which can be simply scripted – much like a2x.py does in AsciiDoc in fact, except that we’re replacing xsltproc with Saxon for greater flexibility (and it also works better in Windows). In this example, I’ll be demonstrating how this works with an example file from the AsciiDoc release, ‘doc/source-highlight-filter.txt’, since it demonstrates the use of the syntax highlighter which is what prompted my deviation from a2x in the first place.

For comparison with the final PDF linked at the bottom, here’s the AsciiDoc input file: source-highlight-filter.txt

Step 1 – run AsciiDoc to create DocBook

Assuming you’re in the AsciiDoc root folder already:

python asciidoc.py --backend docbook  --doctype article  --out-file doc/source-highlight-filter.xml doc/source-highlight-filter.txt

That was easy – you now have a DocBook XML file at doc/source-highlight-filter.xml containing the DocBook representation of your far more friendly text file. a2x usually runs xmllint against this to verify it, but I’m not doing that, especially because it’s not part of my install steps and these Unix-oriented tools require a separate setup process.

Step 2 – Run Saxon to turn DocBook XML into XSL-FO

This step requires quite a lot of typing so this is why you’ll almost certainly want to script it! Replace my “E:/UsefulResources/DocBook” with the real paths you’ve used when unzipping the packages listed above.

java -cp "E:/UsefulResources/DocBook/saxon6/saxon.jar;E:/UsefulResources/DocBook/xslthl/xslthl-2.0.1.jar;E:/UsefulResources/DocBook/docbook-xsl-1.75.2/extensions/saxon65.jar;E:/UsefulResources/DocBook/xml-commons-resolver-1.2/resolver.jar;E:/UsefulResources/DocBook" -Dxslthl.config="file:///E:/UsefulResources/DocBook/docbook-xsl-1.75.2/highlighting/xslthl-config.xml" com.icl.saxon.StyleSheet -x org.apache.xml.resolver.tools.ResolvingXMLReader -y org.apache.xml.resolver.tools.ResolvingXMLReader -r org.apache.xml.resolver.tools.CatalogResolver -o doc\source-highlight-filter.fo doc\source-highlight-filter.xml E:\UsefulResources\DocBook\fo_steve.xsl

If you wanted to make the command-line simpler, you could update your CLASSPATH environment variable with the contents of the -cp option above and then omit it from the command line. Personally I’ve put this inside a batch file anyway so it doesn’t matter to me. Note: on Mac and Linux you should replace the semi-colons (‘;’) separating each element of the classpath with a colon (‘:’) instead.

I’ve called my custom stylesheet, described in the ‘Define a custom style’ section above, “fo_steve.xsl” but of course you can call yours whatever you like and use individual ones per document if you wish.

Step 3 – Run FOP to create PDF

The simplest of the steps, all you need to do now is run Apache FOP on the .fo file created from the last step to produce a PDF:

fop -fo doc\source-highlight-filter.fo -pdf doc\source-highlight-filter.pdf

And the final result should look something like this: source-highlight-filter.pdf

Conclusion

I’m really happy with this new tool chain; the input format is really easy to write and plays nice with source control, and the output quality is extremely good, including language-sensitive code highlighting and rich style customisation. It’s also cross-platform, standards-based, and built on mature yet still modern open source components. This will definitely be the way that I write my technical documentation in future.