Git is picky when it comes to converting large, moderately complex Subversion repositories and so far the only option I’ve found that works reliably is using the very latest version on Linux. Forget about using 1.6.5 on Windows via msysGit, at least for the git-svn conversion it’s very, very unreliable. Similarly I found Git 1.5 on Linux very flaky for the svn conversion. This doesn’t give me the greatest confidence in Git but in order to properly explore all the angles, I’ve committed to making it work even if it means I have to monkey about a bit.
So, I installed a fresh Kubuntu 9.04 (and of course, 9.10 went stable a few days later) and tried to build Git 1.6.5 from source. The configure script is unfortunately a bit rubbish and doesn’t bother trying to detect the dependencies though, so for those that don’t want to go through the fail/retry build loop I went through, here are the packages you’ll want to install via apt from a clean version:
build-essential
curl
libcurl3
tk
subversion
libsvn-perl
cpio
zlib
zlibg1-dev
expat
perl
iconv
If like me you use the excellent wajig wrapper around apt, you can also do ‘wajig build-depend git-svn’, but that seems to install things that are not strictly necessary in the default build (but maybe are needed with some non-standard options).
As I’ve talked about recently, as a background task, I’m setting up a new Ubuntu server to take over the main file server, mail server, build server, backup server, web server, and you-name-it-server duties of my home office. It will eventually be taking over from a venerable Debian server, which was built on some old hardware left over from retired machines (except with 2 new mirrored hard drives) and has basically sat in a corner being rock-solid for years without me touching it, at least until the PSU failed (and was rapidly replaced), reminding me that I’ve been meaning to upgrade it for over a year. This machine does more things than I care to remember half the time, so without a huge amount of time to spare it’s taking me a while to bring its younger, upgraded, more power-efficient replacement fully online.
One of the things I wanted to move away from was my manually scripted backups. Under Linux backups are a generally simpler affair than on Windows; just tar & compress the file system, skipping a few folders like /proc and /tmp and you’re pretty much done, so I wrote a few simple scripts and left it at that, which has worked ok for a while, but as I started to want more complex backup strategies (different subsets sent to S3, etc), manually scripting everything was becoming tedious. I’d tried setting up Amanda before but found it a little overcomplicated for my needs (and reverted to scripts), so I decided to try Bacula this time.
In all it probably took me 3 evenings to get it set up the way I wanted, of which 2 were spent reading the Bacula manual and experimenting, and 1 was solely spent trying to get around an issue with mounting a remote drive on demand. I don’t use tape drives, because I think they’re overpriced and an unnecessary hassle for a small office setup; instead I send all my backups directly to a separate (RAIDed) NAS box, and replicate them to another machine for added resilience. I also upload a critical subset (encrypted) to Amazon S3 for emergency off-site recovery purposes; a removable external HD would be another option, but a) I’d have to keep taking it off-site all the time, and b) you’d really need 2 copies, because removable HDs are just not resilient enough unless they’re at least RAID1).
The problem I had was getting the NAS box mounted on-demand for the backups; I do it over CIFS (Samba) – I could use NFS (since the NAS runs embedded Linux and has NFS support) but the nice thing about using CIFS is that I know I can redirect it to any machine in the office at any time if I need to, since everything supports it (including OS X and Windows), and I won’t start hitting new issues because of a protocol switch. Since we can’t rely on the NAS being mounted all the time (it may be taken offline from time to time), Bacula needs to mount it & unmount it on demand – this is what my manual scripts used to do. Bacula seemed to support this via the “Requires Mount=yes” option on the storage device, and being able to define mount and unmount commands, but I could never get this to work, despite trying for most of an evening (it would just complain about the device not being mounted, and seemingly not try to run the specified mount commands). In the end, I used a much better overall approach anyway, by using autofs to auto-mount the NAS box when a specific path is accessed, and unmount it after a period of no use. That solved the problem in Bacula (I could just point it at a path and let autofs handle the rest), and also meant it’s much easier to browse the NAS from this machine generally.
So, now I have it set up, I’m really quite pleased with it. It supports on the fly compression but does it per file (unlike how you would do it with tar, which is on the whole archive) which makes it a little more reassuring that you wouldn’t lose as much in the case of a limited data corruption. The backup schedules and volume recycling configuration are really very powerful, and they have features in there which make doing backups to disk systems rather than tapes very convenient. The visibility is generally a lot better than my manual scripts, restoring is easier, it’s easier to keep on top of the space used with the recycling, and it’s far easier to manage a ‘deeper’ full/incremental policy without making your restore process harder. Plus, I have a queryable log of everything (rather than just cron logs and notification emails) which feeds metadata into the restore process as you’d expect. Good stuff – all the kinds of features you’d expect from a commercial backup solution.
Since I don’t install X on my servers, I had to configure everything with a text editor rather than the GUI Qt tool (‘bat’), but really it wasn’t a big deal. Once you understood the concepts, the text configuration was really quite straightforward (and well commented), as most good Linux server apps are. Of course, I can run the GUI on my Windows or Mac desktops too, and configure the scheduler remotely from there. I haven’t tried that yet.
So far, Bacula seems like a great solution for small office backups.
I’m busy, again. A ton of things just bunched up towards the end of the month, and I’m on-site with a customer in Cambridge some of next week, so I’m keeping my head down a little right now. Here’s a news-blast though.
I love Ubuntu server
I’ve been setting up my new server. I’ve probably said this before, but for servers, Linux rocks. I’m ambivalent about Linux on the desktop, where I believe consistency and usability are more important (the Mac floats my boat the most there, and Windows if only because of MSVC++), but for a server Linux really brings great things to the table. Rock solid server apps, ridiculously good performance for the hardware (a mere Intel Atom 330, goes like a greased whippet), easy and most importantly infrequent maintenance. When it comes to distros, I loved Debian for its sensible defaults and great package management, but Ubuntu server takes that and makes it even better. The added bonus of a LTS version that I know will be supported with security updates until at least 2013 is welcome too, because I like to set these things up and mostly forget about them.
I’m glad Ubuntu’s default mail/IMAP servers are Postfix and Dovecot respectively – they’re just ridiculously easy to set up. I’d been using Exim4 on my Debian box, which was the default at the time, and learned to dislike it because of the over-complex configuration (I’d been used to Postfix when I ran a Gentoo server before that). I’m also planning on trying out Bacula as a replacement for my manual backup scripts this time around.
PSUs hate me
One of my jobs this week was building a machine around some customer hardware which I was testing an abberation on. Having built it, I realised I didn’t have a spare PSU rated highly enough, so I ‘borrowed’ one out of another GPU test machine I had. It ran all day, then decided to die – this just 2 weeks since my server’s PSU died and needed to be replaced. It’s just bad luck – my decent APC UPS should be providing ample power regulation. So, I’ve ordered 2 new PSUs to make sure I have enough stocked in future!
Excitement is infectious
I was happy to show off some shots of my new & improved core terrain system for Ogre, which isn’t entirely done yet but was usable enough to get some nice shots out of. I knew I had to move on to some other work for a while but I was still pleased to be able to show off some initial eye candy (which BTW, is still very early – I haven’t finished yet by any means). I was glad to get some positive & constructive feedback, but of course now people are rushing off an including it in their projects, since all the code is public in svn. Despite my slapping a big red warning sticker on it saying ‘handle with care – volatile material’ and that they shouldn’t assume it works properly yet, people are hacking on it already, with some nice results I have to say (such as Ogitor integration) – but of course with many questions and issues. Such is the nature of open source – it’s a blessing that you get instant, voluminous feedback, and it’s a curse that you get instant, voluminous feedback I hope to get more time to deal with the fallout from that if not next week (because of my travels), then the week after.
The OGRE Patch Mountain
Our community is always active, and it’s great to get patches. I do have a quite high validation standard for the core though, and processing patches can often take a fair bit of time. I try to spend a few hours per week doing this, but mostly that’s only just enough to keep the level static, rather than reducing the backlog, and it still spikes up sometimes (as it has this week) – that’s because even if I get an afternoon on it, to review and test things properly can eat that up very fast.
If there are any experienced members of the Ogre community who would like to assist me with keeping the patch mountain down to a small hillock in future, and are willing to adhere to our high standards of review, please contact me at sinbad AT ogre3d DOT org.
iPhone 3G / 3GS port coming soon
We’ve had a fledgeling GLES rendersystem around for a while, and obviously the iPhone / iPod touch are the highest-profile targets of that. I’d been intending to have a go at it later in the summer, but masterfalcon on the forums has beaten me to it and already has it running (with a few small issues remaining to be ironed out) on the 3G and 3GS. There should be a public release of that in the relatively near future.
3D Web Browsing With OGRE
I love this video. Nice work princeofcode (aka ajs15822)
I just assembled a new server machine, which in the end I chose to house in a shiny aluminium Thermaltake Lanbox, which is relatively compact but still roomy enough for two hard drives, a bog-standard power supply, and plenty of airflow, which is what I wanted. I also knew that the fans on this case were nice and quiet (I have a black steel version as a GPU test box, I wanted a lighter version this time!), which is important for a machine that will be on all the time.
As I said in my previous posts, I was determined not to put an optical drive in this machine. It really doesn’t need one, since all the system software will be downloaded anyway – the only possible use for an optical drive would be to boot the machine in the first instance, and that seemed a total waste. I know DVD drives are cheap, but why clutter up the box with one just for the rare occasion when I need to manually boot? The same goes for floppy drives, which are such dinosaurs I can’t believe some new machines still come with one present – the only possible use for a floppy drive these days is to provide a slot that you don’t mind a toddler feeding jam sandwiches into.
No, instead I wanted to boot from a USB flash drive. I’d never done this before, so I scouted around for the best ways to do it. Syslinux came up pretty quickly as the primary contender, but being lazy I hunted around a bit longer to see if anyone had a simpler way than configuring Syslinux manually. That’s when I came across UNetbootin.
What a fantastic little project! It took literally 5 minutes from downloading, to creating a bootable USB disk with the distribution of my choice on it (UNetbootin will download your chosen distribution automatically – or you can supply an ISO of your choice if you want), to booting up the new machine. I couldn’t believe just how simple it was! I chose to put Ubuntu 8.04 Netinstall on the disk, which clocks in at a tiny 9Mb because it’s just enough to boot up the installer to start downloading the real packages direct, but if you want, and you have a big enough USB stick, you can a complete distro on there too. But this way, I can use a crappy old USB stick I have lying around as my boot device.
A great little tool anyway. I love it when things are easier than you expect.
I read with some interest Matt Asay’s blog on TWiki, and what has happened over there as the company associated with the open-source project has basically decided to ‘reorganise’ everything, it appears in order to make itself more attractive to venture capitalists.
To be honest, I really don’t understand the motivation at all. All open source projects live or die by the strength of their community, and to suddenly break from it in the interests of attracting investment is crazy. Personally, I’ve never wanted to take VC money if I can help it – I’d prefer to run a small, self-funded and organically growing ship that I can stay in control of, and which I can apply my own brand of ethics to in balance with the need to make a living. Balancing open source and commercial necessity (we all have to eat after all) is tough, and it’s very different to running a regular proprietary software business, so you really can’t apply the same rules without undermining the very basis of the business.
Unfortunately open source poster-child examples like Ubuntu don’t help in many ways. Ubuntu manages to do everything the ‘right’ open source way while still having gazillions of dollars to spend on premises, staff, servers etc – but that’s only because it’s backed by an interested billionnaire who doesn’t really care how long it takes to turn a profit (and probably wouldn’t be too fussed if it never did). So in some ways Ubuntu makes it hard for others because it’s often held up as an example of how things should be done, when in fact almost no-one else can afford to do it that way, unless they can find a billionnaire of their own. Perhaps that leads to cases like TWiki for some projects, where ‘Ubuntu envy’ leads them to chase investment, but at the detriment of the reason they exist in the first place.
All I can say is that this will never happen to Ogre while I’m in charge. I obviously have to seek commercial opportunities related to Ogre, but I have a very deep line in the sand drawn many moons ago that I will never cross. At times people have asked me ‘what would happen if Ogre got acquired?’ – and I have to patiently explain to them that even if I wanted that to happen (and I don’t), it’s actually not possible in a traditional sense, since my company doesn’t own all the code. It owns a lot of it for sure, but the rest is community-contributed and licensed by TKS based on the contributor agreements – which in our case don’t ask for copyright assignment, just permission to use & relicense. This means that no-one could come along and ‘buy it up’, or at least not in the traditional sense. They could buy the domain from me I suppose, and the rights that TKS has, but could not fundamentally change the licensing conditions without approaching the contributors for permission. It’s commercially resticting, but I also see it as a key factor in reassuring the community about the intentions of my company.
When it comes down to it, in many ways having this restriction there is unnecessary – even if I was able to ‘sell’ Ogre, or suddenly change the licensing, I would be stupid to do it, because it would immediately destroy the community, which is what has made it great. Someone would fork a new project from it, and with a bit of time, that would become the ‘standard’ version. There might be some opportunity to ‘milk’ the codebase with custom commercial versions for a little while, but it wouldn’t last. The whole idea is self-defeating in the medium to long-term, as TWiki.net will probably discover shortly.
I’ve talked about business models and open source before, and that it can be necessary for companies like mine to mix in some proprietary aspects sometimes (e.g. optional add-ons) to make ends meet. However, maintaining the absolute integrity of the central open source project and its community at all times is absolutely vital. That’s the heart of it, and any business destabilises that at its absolute peril.
I happened to be passing through one of our local Fair Trade shops today to pick up some more coffee beans, as is my habit – not only do they have more variety than most regular supermarkets, but their blends are almost universally better quality, thus slaking the taste buds of the discerning Java drinker as well as giving a warm fuzzy feeling that you’re doing something positive for farmers in developing countries, or at least helping them get screwed slightly less than they would otherwise be.
Anyway, the crux of this story is not coffee, because right there on the counter as I was paying I spotted a drinks can emblazoned with the name ‘Ubuntu Cola’, which made me suddenly smirk and chuckle involuntarily in a way that prompted the cashier to give me an unusual look. Of course, I could have spent the time explaining to him that Ubuntu was the name of a free software operating system, and thus to see it turning up on the side of a can of cola was pretty surprising, but we could have been there for a while, and he probably wouldn’t have thought me any less crazy by the end of it either – I suspect he would have just been a little more specific about it (i.e. a crazy geek). As such it was faster just to grab a can, add it to my tab and be on my way.
So what’s it like? Well, it’s a little darker than regular big-name cola and tastes quite a bit like Pepsi, although oddly it seems to remind me of the Pepsi of my youth, perhaps because the ingredient list seems a little simpler and it’s loaded with natural cane sugar rather than ‘high fructose corn syrup’ (whatever that is). Overall pretty good – I usually don’t choose to drink cola these days but it was worth the experiment, if only for the blogworthiness
As it turns out, there’s a whole Ubuntu Trading Company and it appears Ubuntu Cola might just be the first of many Ubuntu food products. I wonder what Mark Shuttleworth thinks of that?
I might be misunderstanding, but it appears that the core of the grassroots Tory activist policy is for it to be 1955 again - 3 days ago
Apparently my aversion to baggage while travelling is 'hobo-esque'. The beard probably doesn't help. #whoneedsrazors - 3 days ago
Reminder: Guernsey Tech Tribes meetup is on 30th May: lanyrd.com/2013/stevestre… - please register if you're coming, you can use your Twitter id - 4 days ago
I hope to get more time to deal with the fallout from that if not next week (because of my travels), then the week after.
I happened to be passing through one of our local 
Home