Search Results for: mercurial

Business Cocoa Development Objective C OS X Personal

SourceTree, your Mac Git & Mercurial GUI, is 40% off this week

Since I’m trying to spread this news as far and wide as I can, I might as well say it here too :)

Since the approval light just went green on the Mac App Store, I’m happy to announce the launch of SourceTree 1.2! In celebration, I’m having a crazy-bonkers 40% off sale just for one week, so get it while it’s hot!

There’s loads of things that are new or improved in this release, but here are the headlines:

  • Support for GitHub, Bitbucket and Kiln APIs, so you can see your hosted projects inside SourceTree, clone from them, link them as remotes, and even create new projects if you want.
  • Streamlined and polished user interface – I specifically dedicated a lot of extra time in this release on making SourceTree easier on the eyes, and to streamline the layout and workflows better.
  • Performance - I thought SourceTree was already pretty fast, but I managed to find quite a few more places to trim the fat, and also parallelised more activities to make things feel more responsive. Everything feels snappier, and complex repositories benefit especially.
  • New Sidebar - I had previously resisted the need to emulate iTunes here, but once I had implemented it, I had to admit that I was wrong, and in fact this worked great. Provides lots of shortcuts to navigating and operating on branches, tags and remotes.
  • Stashing and Shelving - oft requested, now delivered :)
  • Customise Git and Mercurial – you can now use your system Git / Mercurial instead of SourceTree’s standard versions (which have been updated), and enable additional Mercurial extensions (at your own risk).
  • French and Japanese translations – local versions for our friends in far away (and not so far away) places, likely to be more to come in future. Big thanks to tuan_kuranes and mzch for their help with these two!
  • And the rest – just lots of little refinements too numerous to list. Examples: copying text from the diff panel, ‘git commit –amend’ support, close branches in Mercurial, switch tracking branches in Git

It’s quite a big update – one user remarked to me that they’d normally expect developers to charge an upgrade fee for something like this, but like all other SourceTree updates this is free to existing customers. I have no plans for any paid upgrades for some time yet, I just want to keep making SourceTree better, and hope that more people come onboard. Maybe it’s my open source background, but I like to keep iterating and continually improving things, based on what I want to do (I’m a daily SourceTree user myself), and on what people tell me they’d like to see. SourceTree 1.2 certainly won’t be the last update by far :)

When I look back 6 months at SourceTree 1.0, it’s incredible how much better it is as a product now, both visually and functionally. I’ve learned a ton of things while I’ve been developing it, and I continue to learn more all the time, and I can’t think of anything I’d rather be doing right now. Also, my wife Marie re-designed many of the icons for 1.2 (and I think you’ll agree they’re a lot nicer) – that was fun to do as a joint project, even if I am a picky ‘customer’ ;)

I hope you enjoy the new release!

Development OGRE

Mercurial queues just clicked

When I was evaluating Mercurial, I touched briefly on Mercurial Queues (MQ), but at the time felt that they were probably a little more complicated than I needed. After all, one of the things I liked about Mercurial was that it all felt rather straightforward and intuitive, while MQ seemed downright confusing. However, I’ve just discovered a really powerful use case that has meant I’m now committed to using MQ for my every day workflow. But I’ll come back to that.

First, some explanation of why I revisited this. I’ve been working with Mercurial in a live situation now for a few weeks now, and my experience has been uniformly good. Particularly the main thing that used to be a total pain in my ass before – merges – has become infinitely less frustrating, and I even managed to use it to take a client’s highly modified version of Ogre which was forked from quite an old base, and bring it up to date with later versions via 3-way merges in a fraction of the time it would have taken me with Subversion’s relatively brute-force, roundtrip-generating merge system.

However, one thing that has come up is that some other team members have found the ‘uncommitted changes’ error message annoying – that is, there are some things that Mercurial won’t let you do if you have uncommitted changes; mainly merge and rebase, since both of these work on committing (or committed) changesets. Some of this confusion has been caused by me telling people to always use ‘hg pull –rebase’, to avoid unnecessary merge points when parallel development has happened, but the knock-on effect is that using the ‘–rebase’ option makes Mercurial stop if you haven’t committed your current work.

Resolving This Without Mercurial Queues

  1. Finish what you’re working on and commit it before pulling new changes (with –rebase). In fact, most of the time this is just fine, unless for some reason you’re dependent on someone else’s work to finish yours. Some people got into the habit of pulling SVN changes every morning regardless of whether they’re in the middle of something or not, but actually you don’t need to, and in fact doing so can introduce unexpected and unwanted variables into your existing work. But, there are some times when you genuinely want to pull other people’s changes before you’ve finished, so..
  2. Drop the ‘pull with rebase’ and use a regular ‘pull with update’ instead, ie “hg pull -u”. This is compatible with outstanding uncommitted changes – however if you’ve also got unpushed changesets of your own locally, bear in mind you’ll end up with multiple heads (branches) after you pull, and you’ll need to remember to call “hg rebase” directly (instead of as part of your pull) before you push next time to unify them.
  3. Use “hg shelve” (extension) or simply ‘hg diff > somepatchfile’  to stash your uncommitted changes, then pull –rebase, then unshelve  (or import / patch) your changes in again. However, if you’re considering this option I urge you to try to use MQ instead.

Mercurial Queues – juggle many unfinished changes at once

The important thing to understand about MQ, which I didn’t grasp at first, is that it’s basically just designed to automate the process of using ‘hg diff’ to generate named patch files which you stash away somewhere as an unfinished change. I used to do this all the time in CVS and SVN when working on multiple things that weren’t quite ready to be committed yet – dump my current changes to a patch file, put it on a server or USB stick (so I can apply it on other machines & back it up), revert my changes then start work on something else, pulling the patch file back again when I want to re-start it. You can still use this manual approach with Mercurial of course, but this is also exactly what MQ does, it just has a bunch of explicit commands for it, and keeps track of which patches you have applied already in your local copy.

For example, ‘hg qnew mypatch’ creates a new patch and imports your current uncommitted changes to the patch (after which your ‘hg status’ shows as unchanged since hg is MQ-aware). It’s actually created a patch file of the changes in .hg/patches/mypatch. If you subsequently make changes, hg status shows the changes since you did the hg qnew, and you can update the patch to the latest set of changes by using ‘hg qrefresh’ – note how this just updates that patch file, it doesn’t create any changesets or anything.

If you want to go back to the status of the repo without this patch applied, just call ‘hg qpop’, or ‘hg qpush mypatch’ to bring it back. The reason for the ‘push’ and ‘pop’ concept is that each patch (which you create with qnew, you can call it at any time to spawn a new patch with just your current alterations) is considered to be in an ordered stack, with each one depending on the one underneath (and applying the patch at the top of the stack implies applying all those beneath it too). You can navigate around the stack whenever you like, qrefreshing individual patches (remember, they’re not changesets so they’re mutable – this is how you get around the fact that Mercurial’s history is not changeable), but you’re always in a stack. This is the one thing I’m not so fond of about MQ, I think patches should be in a list of stacks to allow you to have unrelated (and potentially breaking) sequences of patches tracked differently in parallel. The sharp-minded among you will note that git’s equivalent – which is to use local branches – effectively allows this by nature of the branch structure. If you’re really in need of this behaviour, you could try the LocalBranch extension or use local clones instead of MQ.

However, there’s something else I particularly like about MQ over the local branch approach, and that’s what happens when you want to synchronise your work-in-progress changes with other machines (or other people).

Repositories within repositories

When using local branches (in Git or via LocalBranch), you’re always working with regular committed changesets. They’re not pushed to others by default, but if you want to synchronise them you have ways to force them to push (or pull) – for example when you work across multiple machines. You might do this direct, or if the machines are not on at the same time you may do it via an intermediate server or USB stick. The main problems I have with this approach when related to work-in-progress changes are:

  • Inherently these changesets are not finished, and you’re not going to publish them as-is elsewhere. In Git you can collapse them together later of course with the finished state, but it means that even locally you’re synchronising unstable state, which is fragile if you screw up
  • If you need an intermediary for the sync (server or USB stick), it has to be a full repository clone even though you only need your WIP changes.

Of course, you could generate patch files yourself in both cases to remove both of these issues. But that’s why MQ is ideal, because it already does that for you, without you having to track external patch files. And best of all, it has a simple way to synchronise and version control just those mutable patches with others, without ever raising the issue of history modification: it simply creates a Mercurial repository within the .hg/patches directory! This repository only contains your MQ patches, and can be safely published, synchronised without ever looking like a proper ‘history’ of the main repository. Best of all, most of the regular hg commands take a ‘–mq’ option to perform actions on this nested patch repository instead of the main repository.

Putting it all together

Since Mercurial 1.5 you haven’t strictly needed to call ‘hg qinit’ to start using Mercurial Queues, but in order to create a nested repository for sync, you still want to call it, like this:

hg qinit -c

That will create a nested repository in .hg/patches which maintains its own history and can be synchronised with others, independently of the main repository. You might want to take the opportunity to edit .hg/patches/.hg/hgrc and provide a default remote path if like me you want to specify a server you will use to keep all your WIP patches for this repository.
So let’s say you’ve been doing some work on a change, but it’s not ready yet and you want to put a stake in the ground where you are without committing. Simple, you just do:

hg qnew awesome-feature-1 -m"I'm working on an awesome feature"

That says “create me a new patch in .hg/patches/awesome-feature-1 with my current changes, and give it an initial description” – you can change this description later whenever you want when you update the patch with qrefresh, see below. At this point hg considers that you have no local uncommitted changes anymore so you can happily do rebases and merges without the ‘outstanding uncommitted changes’ message. But, there’s more, you may want to do the following so you can sync the current state of this patch with other machines:

hg commit --mq -m"My awesome feature WIP #1"

Woah, woah, I said you didn’t want to commit, right, because this is a work in progress? Notice though, that I called commit with the ‘–mq’ option, which means ‘commit changes to the MQ patches repository’, and not ‘commit working copy changes’. If you look now, you’ll see that you have a patch file called .hg/patches/awesome-feature-1, which was automatically added to the nested patch repository when you called ‘qnew’, and which was committed to that repository in its current state (remember, this patch repository is completely separate from the main repo).

If I wanted to push the current state of my WIP patches to somewhere else, I can do it now (I’m assuming that I’ve already edited .hg/patches/.hg/hgrc to include a default remote path:

hg push --mq

That pushes the commits I’ve made to the patch queue (not the main repo) to that remote location. So for example I use a small area on a local server to track this, but there’s no reason I couldn’t push this somewhere public too if I wanted – again all explicitly separate from my actual repo, which is perfect. All the other main commands take a –mq option too, so you can use “hg outgoing –mq” etc too.

At this point, I could carry on working on my same patch, and every time I wanted to update the patch file in .hg/patches with my local changes I would call:

hg qrefresh

You can use the ‘-e’ command if you want to update the commit message that will eventually be used. And again, if I wanted to commit the changes to that patch, I’d call ‘hg commit –mq’ (and potentially ‘hg push –mq’). If at some point I wanted to quickly work on a different change, I could call ‘hg qnew’ again to create a new patch file based on the current diff, and deal with that separately. To navigate around the patch queue to identify which patch I’m currently working on, I’d use qpop and qpush.

Finally, at some point I’m going to want to turn these patches into real commits. Assuming that my commit message is already correct in the patch (if not, use ‘hg qrefresh -e’ to edit it), and that your current position on the patch stack is the position you want to create a proper commit for, all you need to do is:

hg qfinish --applied

Which will move the patches you currently have applied into the real Mercurial repository history, ready to be officially pushed with everyday work. Alternatively if you want to just pick a revision or range of revisions, you can name them instead of using –applied.

Conclusion

I didn’t really see the advantage of using MQ over just keeping my own patch files generated from my working copy via ‘hg diff’, particularly since MQ makes you work in a stack-based structure instead of via independent patches, until I learned how to use the nested patch repository. Manually handling patch files is fine, but once you get more than a couple of them on the go it can get tricky to manage. Having an automated approach to this with the nested repository works really very well, so I’m now using it as my primary method of managing work-in-progress.

Git users will no doubt think the local branch approach is better, and in at least one way (not having to keep patches in a single stack), it is – however I personally like how MQ mirrors my normal approach to unfinished changes as ‘volatile patches’ rather than treating unfinished changes as regular changesets and relying on history modification to fix them up later, and as someone who regularly moves between machines, MQ’s ability to sync patches separately from the main repository is very valuable. As is ever the case with Mercurial and Git, the perfect solution would probably be some combination of the two.

Development Tech

Early-stage Git/Mercurial/Bazaar evaluation thoughts

A few weeks ago I decided to start seriously investigating switching to a DVCS. I’m currently up to my eyes in work and haven’t really had time to progress that in the last few weeks; however some absolutely abhorrent performance / reliability problems with Sourceforge’s Subversion server made a large merge process so costly to me (in the end I had to commit in small chunks, breaking transactional consistency, and it needed so much babysitting because of the speed / reliability it took me 4 bloody hours just to commit!!) that it bumped it up my agenda a bit. I don’t have to do merges / commits of that size very often – in this case the problem was wholesale license header changes for our MIT switch – but still, it’s totally unacceptable to have to deal with that. I raised a support request with Sourceforge, but I’ve seen other reports of bad SVN performance from several weeks ago from others, so I’m not holding my breath. It’s time to start considering alternative hosting I think.

I’m not done with my evaluation yet, because I just don’t have the dedicated time I really want to spend on this right now. But, here’s my early-stage results.

Git

I’ve discussed before that I don’t like the where Git has come from. It’s overly complicated, Windows support was clearly not a priority, and it switched existing VCS terminology around just for the sake of it a lot of the time. It practically shouts “I’m different, live with it!” at you, which is frankly a typical elitist geek attitude and not one I particularly respect. This attitude permeates the documentation, reinforced by the absolute insistence of most tutorials that you need to understand Git’s underlying data model before you start using it. Er, right – sorry, but when normal people want to learn how to use a new tool, we just want to know how to use it, not how it’s built. If understanding how it’s built is a prerequisite to using it, then I’m sorry, it fails miserably at being user friendly.

Nevertheless, it’s fast, it’s efficient in storage, it’s the most popular & fashionable DVCS (probably due to GitHub) and that has weight. Of particular interest to me is that TortoiseGit has come along in leaps and bounds, and is really quite pleasant to use. Of course, the reason it’s pleasant is because it hides the majority of the nonsensical changes that Git decided to make to existing VCS terminology; for example ‘Revert’ in TortoiseGit does what you expect (undoes your working copy changes), rather than needing to use ‘git reset –hard’, which is only intuitive to those who live on Mars (git revert, in contrast, records a new commit to undo a previous commit – why the hell do you need a special command for that??). Thus, it’s an odd situation – using TortoiseGit is pleasant, but only because it ushers the underlying git behind a curtain and gives you what most people really wanted from it in the first place. The downside is that using TortoiseGit really doesn’t teach you how to use the command line very well, like most other VCS tools do. In fact, it may well mislead you into thinking Git is friendlier than it actually is. For example, it saves you from the ridiculous need to remember the “-a” argument to “git commit” – without which what you actually get in your commit is the state of the file when you did “git add”, not the version in your working copy. If that makes sense to anyone, raise your hand. Thought not.

Hosting & collaboration wise, GitHub seems very good.

Mercurial

Mercurial on the command line is nice. It behaves the same way centralised VCS’s do, except in the cases where it needs to be different. This is pragmatic design – not being different just to make a point, but being different where it needs to be. It doesn’t break old concepts and does what you expect it to, and contrary to what some people think, that’s a very valuable feature.

It’s not all roses though. TortoiseHg is clunkier than TortoiseGit, despite being based on a more intuitive core tool. The UI just feels a bit wrong (like putting action buttons on the toolbar – who does that?), and I’ve sworn at it for being unintuitive more than once. The other problem is that the Mac GUI tools are not really that great either – MacMercurial only allowed me to do a subset of the operations I needed to do, and Murky just crashed when I tried it. GitX in comparison works quite well on the Mac.

So despite a more intuitive command line and core concepts, and a more pragmatic approach generally to DVCS for ‘regular’ people, when it comes to GUIs Mercurial lags a bit now. This was unexpected to me since it is Git that has traditionally been poor on the GUI front. There are also a few other minor issues like branches being totally permanent and needing to be globally uniquely named, which can make local experiments more cumbersome.

Hosting wise, BitBucket seems quite competent, if a little less polished than GitHub.

Bazaar

I’ve only just started experimenting with Bazaar, and so far I’m quite impressed. It has the pragmatic approach of Mercurial, but also has a built-in GUI which is really quite nice to use and leads you through the initial setup and configuration. There’s also TortoiseBzr which feels somewhere in between TortoiseHg and TortoiseGit. I haven’t tried it on the Mac yet. Performance was always the issue listed as the major downside of Bazaar, but this has improved since 2.0 and while it’s not as fast as Git, it seems to be fast enough.

The main downside for Bazaar is adoption. It trails both Git and Mercurial in terms of the number of people using it, and therefore adopting it for a public project would have the disadvantage of making people use a tool they’re less likely to already be familiar with. Also for hosting, Launchpad is quite new; it looks quite good, and has more features than GitHub, but it doesn’t have the option to host private projects (not an issue for Ogre of course) or a graduated commercial plan – you can self-host of course but that’s not as easy.

Conclusions so far

The sad fact is that none of the 3 are an instant win for me; they all have positive and negative aspects. Summary so far:

Tool Pros Cons
Git Fastest & most efficient
GUIs actually good
Popular
Command line overcomplicated & unintuitive
Mistakes easier to make
Mercurial Intuitive
Fairly popular
GUIs a bit rough in places
Bazaar Very intuitive
Built-in GUI good
TortoiseBzr also good
Not very popular
Statistically the slowest
Launchpad is quite new

So, I’m basically in a no-win scenario. If I pick Git, it’ll work fine via the GUIs but it’s too easy to screw things up when using the command line, and I’m bound to get annoyed at the needless obscurity from time to time. But, lots of people will be happy to use it. If I pick Mercurial, I’ll be happier with the overall core concepts & command line, but the rough edges on the GUIs are going to annoy me day to day. But, quite a few people will be happy with it all the same. If I pick Bazaar, I’ll be happy with both the core concepts and the GUIs, but being the least fashionable option almost no-one in the community will be happy that I picked it over the other two, and lots will bitch about having to use another tool.

I’m reluctantly acknowledging that the least of the evils appears to be Git right now, even though I personally hate its underlying interface. Somehow it feels wrong to only like using it when it’s hidden beneath a GUI – I’ve been a regular user of the command line for CVS and SVN for the best part of a decade, and I like being happy with both modes. I can imagine tolerating Git’s command line, but never liking it just because of the unnecessary idiosyncracies (like commit -a).

I still have lots more detailed tests to do anyway, which will have to wait a month or so until I have more time.

Development Open Source

Playing with Mercurial

I’ve been interested in DVCS for a while; having done my fair share of branch management, something which makes that process easier and more transparent is definitely very attractive. I particularly like the way a DVCS makes it easier for people to collaborate in pockets of their own, away from the centralised environment, and track other repositories and keep their local mods up to date more easily – public-branch-on-demand if you will. However, I’m yet to be convinced by Git for everyday use. As noted, I love the idea, but Git comes across as – and this is an ironic thing for me to say – excessively geeky. At one time I might have believed there was no such thing as a plethora of geekery, but I like to think I’m a more rounded individual now ;)

Don’t get me wrong, I’m sure Git is technically excellent. But a number of things about it remind me of the somewhat stubborn purisms of  ‘old Linux’, such as:

  • The documentation is hugely daunting. Whenever anyone tried to convince me why Git is amazing, they point me at documentation that just makes me not want to touch it for anything important. The impression I get is akin to some Linux forums: – this thing is powerful, but it gives you plenty of rope to if not hang yourself, then to tie yourself in knots if you don’t know what you’re doing.
  • It requires a Unix back end, to the extent that you have to run it under Cygwin on Windows, and its rigid filesystem requirements mean it won’t work on FAT or Windows network drives. I get the impression there’s something rather aloof about the fact that the core team don’t consider it an issue that Git won’t run on native Windows environments. There are external projects working on that, but the fact that it’s not core is a concern, since I prefer all my 3 platforms to be supported equally.
  • The UIs still don’t seem very user-friendly, and most responses I see to the question of why that is boil down to “Git is too powerful to be captured in a GUI”. Sorry, but I don’t buy that – no system is too complex to be captured in a user-friendly tool, it just takes the will to do it, and like ‘old Linux’, there seems to be little will to make Git more approachable.

So, I’ve never had great vibes from Git, despite it’s doubtless technical merits. So, I decided to check out Mercurial instead – from what I read, Git is the fashionable DVCS that all the cool kids want to be seen using, but when it comes down to it Mercurial does exactly the same thing – except that it has native support for all platforms out of the box and TortoiseHg is looking pretty mature. I also read that it handles binary files better than Git too, since it does binary diffs everywhere. The documentation seemed much more approachable too, so I figured I’d have a play.

My first impressions are incredibly positive. The most impressive thing of all is that I imported a (relatively small- about 100 revisions) Subversion repository, with all the history intact, about 5 minutes after installing it, and was bouncing that across Linux and Windows immediately afterwards (haven’t tried OS X yet). TortoiseHg is instantly familiar, and despite the fact that the concepts are the same as Git, the presence of some familiarity, together with some distinctly less intimidating documentation, has me feeling far happier than the times I’ve dipped into the Git docs. Mercurial’s approachability is a positive contrast to Git’s Unix-purist, RTFM style I think. I know which I prefer so far.

Development Personal

[self.sourceTree exit];

Four and a half years ago, I decided to write a Mac tool for Git and Mercurial, which I’d eventually name SourceTree (aside: names are hard, and I was quite pleased with this one). I wasn’t happy with the Mac apps that were out there at the time and thought I could write something that fit my needs better, and by extension the needs of other developers who felt like I did. I’d never written a Mac-specific app before, and I thought it would be fun to learn how. I also knew that the Mac was a platform where independent app developers could make a living on ‘premium’ apps, so it made business sense. I could see the beginnings of the trend away from Subversion to git/hg and felt there would be an expanding market for such a tool. So, I took a punt.

With a little apprehension I quit all my contracting gigs, bought a couple of books on Objective-C and Cocoa and got my head down, trying to ignore the dwindling bank balance as the months passed. The words JFDI, Lean Startup, Customer Development all rang in my ears. 6 months later, and almost exactly 4 years ago today, I announced the launch of SourceTree version 1.0 on this blog. It was a slow start, but I was prepared for that, and in many ways it was a boon because it gave me time to refine the product with a small but growing gang of early adopters. I took a little more contracting work on the side for a few months to stabilise my finances while I iterated, but only 5 months after that initial release, SourceTree passed an inflection point and adoption started climbing – it was in profit and growing at a good speed. Around this time Atlassian approached me about an acquisition; at the time I was happy to just have a profitable product that meant I could afford to work on what I wanted, but they were convinced that together we could take this product to a whole other level. I eventually decided to accept, and boy, were they right.

The blur of the following 3 years included: porting SourceTree to Windows; hiring fellow Guernsey developer Kieran Senior whom I mentored with all the subtlety of the Eye of Sauron; morphing into a team spread across Guernsey/Amsterdam/San Francisco/Sydney with a real designer, support and marketing folks, and above all, watching a ton more people start to use SourceTree. More than I could ever have imagined. When I stepped down from Ogre I wondered whether I’d ever build anything that popular again, but it turns out the answer was yes, and then some. It’s been a pleasure, and my thanks to everyone who helped make this possible.

However, as much as I love SourceTree and have enjoyed this amazing journey, it’s time for me to seek a new challenge. Four and a half years is a long time to work on one product, and there are so many other problems I’d like to solve. We hired Mike Minns earlier this year to lead the team, he’s a seasoned lead developer located in Sydney (and ironically an ex-pat – we don’t deliberately recruit everyone from the British Isles, honest!), and together with Kieran will be responsible for all future SourceTree development, while I bow out and take a merely advisory role when needed – which will likely comprise of ‘Steve, what the f*** is this code??’ ;).

I’m starting work on one of the (many) projects I shelved a while ago – unsurprisingly it’s one of those problem areas that bugs me personally (what else?) and I’m pretty certain that other people will see the merit in addressing it the way I have in mind too. Hopefully :) I’m still with Atlassian, because they’ve been great about giving me freedom and this particular project very much suits doing within this environment. I’ll talk more about this when the time is right of course. *tease*

I’m posting about this on my blog because a number of people know me personally as the go-to guy for SourceTree, and sometimes short-circuit the official reporting channels for feature requests and bugs on Twitter and email. This was fine, I always like to connect personally with people using my products, but obviously in the future this won’t work any more. Thanks for all the feedback and support over the over the last 4 years, I’m so lucky to have managed to make another tool that developers really connected with. SourceTree will continue without me and I’m sure we’ll see great things from the team in future. Now I’m off to see if I can do something else useful :)

Internet Local

Guernsey broadband should aim to lead, not bring up the rear

snailI was invited to write this blog post by @cutoffgg, a group raising awareness of how poor the broadband options available in Guernsey are, and I readily agreed. Everyone I talk to in Guernsey, whether they be a businesses or home user, has something to complain about when it comes to the Internet service they receive here for the price they pay, yet the our providers continuously give us the impression that should be grateful for the ‘competitive’ service we receive. I’d like to illustrate how disingenuous that is based on my experience over the last few years, and why that’s a problem.

I’m a software developer, a technophile, and a local. I like making tools, engines, applications and other bits of software kit which scratch my various itches, which most recently have been real-time graphics and developer tools. In previous years, being unwilling as I am to join the rank and file of our local finance industry would likely mean I’d have to leave for more suitable climes, but the Internet changed all that and allowed me to remain here to indulge my sentimentality and keep my wife happy.

So, I’ve worked from home for the last 7 years, and have spent that time freelancing for companies around the world and also building my own software products for the global market, one of which end up being acquired (the obligatory ‘look Mum, I’m on TechCrunch’ link). During these 7 years, 99.9% of my customers and co-workers have been outside Guernsey. I’ve sold software in 40 countries and worked in teams which regularly cross 3 or more time zones – we use an array of tools (video chat, persistent text chat, rich wikis, forums) and working approaches (self-motivation and trust is key) which make this manageable – although meeting in person is still necessary and desirable sometimes of course. Still, distributed working and instant access to a global audience regardless of my location on a small dot in the ocean is just ‘normal’ for me by now, and I can’t imagine working any other way.

The Internet is absolutely vital to opening opportunities, and it’s an incomparable access port to the world which is especially vital for small, isolated places like Guernsey; and yet our connections here are over-priced and under-specified, lagging behind most other places in the developed world. When it comes to competing on the world stage, we should be aiming to have some of the best connectivity available in order to balance out the disadvantages we have due to our isolation and lack of land area. And yet,  in all the distributed teams I’ve worked on in the last 7 years, my connection has always been slower and more expensive than anyone else’s on the team.

This is always most pronounced when it comes to upload speeds. Internet providers seem to assume we’re mostly just consumers of information, and that we hardly ever upload. Newsflash guys, the Cloud happened, YouTube happened, online backups happened, and if you’re a digital exporter like me, upload speed is extremely important. I have years under my belt of being that guy who pushes the red button to launch new releases to that global audience, and we all sit there drumming our fingers while the progress bar creeps languidly upwards.

OK, so that’s the background, let’s talk specifics. I currently pay £50pm for a ‘Pro’ ADSL connection from Sure. I pay this crazy price for a ‘Pro’ version for the promise of a lower contention ratio and for a marginally increased upload speed, which tops out at a staggering 1Mbps (which is still about 50% faster than the non-Pro). The downstream rate is advertised as 20Mbps, but actually delivers 7-9Mbps in practice. My house is 20 minutes walk from the centre of our main town, so I’m not exactly deep in the wilderness, but the nearest fibre cabinet to me is just under 2km away, which is why I get such a poor connection. Our monopoly infrastructure supplier apparently has no plans to change that in the next 18 months, meaning the whole area south of our main centre is poorly supplied. While the supplier makes an alternative VDSL option available, which looks better on paper, but in practice unless you’re basically sitting on top of a DSLAM performance is no better (and sometimes worse, judging by some reports I’ve heard).

I also kinda thought I’d get an improved customer experience with the ‘Pro’ package too, but actually I still get 2-3 unexplained disconnections a day. Thanks guys, really helps when you’re on a video call with your co-workers, or you’re in the middle of a release going out to a hundred thousand people.

I did a quick poll to compare the practical experience friends in my industry are getting in other countries with their personal connections. Below are the results from people who answered in the time window; these are not numbers plucked randomly from marketing materials, nor are they ‘limited’ plans, the usual response I get from telcos here:

Location Monthly Price Download (Mbps) Upload (Mbps) Comments
Me £49.99 20 (really 9) 1 My Pro ADSL. I could reduce the price to £25 if I went non-Pro & get the same speed in practice. Advertised rates for VDSL seem unachievable for most.
Singapore £25.85 150 70 And yes, this is his real achieved speed and not just the advertised rates!
Vienna, Austria £21.51 50 5 Occasionally (5% of the time) rates drop well below advertised, otherwise OK
Toronto, Canada £78.56 150 10 Canadian telco duopoly makes it expensive, but speeds are far above what we can get
Wroclaw, Poland £19 20 1.5 The closest to ours so far, but still cheaper
Jutland, Denmark £19.50 30 30 I want to live in Denmark
Maryland, USA £54.24
£88.82
32
150
8
65
Kinda expensive for the USA but still better than ours, especially upload, and scales up fast
San Francisco, USA £39.22 50 10
Carrickfergus, N. Ireland £26 80 (really 45) 10 'the tin-pot local Northern Ireland' – his words not mine :)

Even if you discount my connection down to £25pm by dropping the ‘Pro’ option (which in practice I’m not convinced is benefitting me at all), Guernsey is right at the bottom of the pack. Singapore and Denmark have just awesome deals, and most other countries are in the middle somewhere. Only Canada has the same sort of over-pricing that we see in Guernsey, for similar reasons (not enough competition in infrastructure) and even then the speeds they have access to are far in excess of ours, and they have far more serious distances to cover. While many of these results are from cities, not all are and you only have to look at the expanding fibre coverage in the developed world to see why we’re getting left behind.

Seriously, we must aim to do better than this. We’re an isolated community with limited physical space and connectivity is a key to both staying in the games we’re already in, and providing escape hatches to finding new ones. The regulator CICRA is currently conducting investigations into business broadband provision, which is great, but I hope they also consider home and ‘prosumer’ connections at some point, because those are the ones used by freelancers and small independent creators, who are a growing part of the economy. Great communication links offer opportunities for lightweight diversification without creating more population issues, and gives easier access to remote expertise on demand. The old assumption that the only way create a new business is to squeeze a load of bodies into one place is long dead. Wake up and smell the coffee, people. Jersey has ;)

Personal

On pigeon holing

If you haven’t come across them already, I strongly recommend you take a few minutes with this HBR blog: In Defense of Polymaths, and also Adam Savage’s commencement address to Sarah Lawrence. Both are insightful pieces on the fallacy that is the tendency to believe that specialism in a narrow field is the answer to a fulfilling life experience, and ultimately to ‘success’, whatever that means – usually money, possessions and peer recognition. I thought I’d add my 2c to the conversation.

I spent a lot of my youth rather confused about what I wanted to do, because there were a bunch of things that I was either good at or had an interest in, and those things were often in flux, yet I felt the expectation that to succeed, I had to pick one and specialise in it for the foreseeable future. Life has taught me that this is just nonsense, and that unless you have a singular all-consuming passion that lasts for 40 years, you should expect to switch things up sometimes, at least if you want to stay happy.

I’ve switched up my ‘career’ (if you want to call it that) lots of times already and I expect to keep on doing it. Straight out of school I trained as an accountant before realising that didn’t interest me, but came out of it with some business, law and economics skills that I still use occasionally. I’ve trained on ‘big enterprise systems’, and for a while sought out the big, hard, complicated design problems. I also did a lot of graphics coding, from software engines to hardware accelerated, 2D and 3D systems, and ended up creating an open source project in this space that became very well known. I’ve written Mac software, currently in the developer tool space (git and mercurial). All of these things continue to be useful to me in some way, regardless of the tangental association with whatever the current project is. The Steve Martin quote that Adam Savage uses in his speech resonated with me:

You will eventually use everything you ever learned. ~ Steve Martin

However, every single time I’ve made a leap from one thing to another (I call them my ‘context switches’), I’ve had people say to me ‘But I thought you were the guy who did X!’. When I went from enterprise systems to graphics software, people thought this was odd – why wasn’t I following the natural progression (and job offers) from the previous 10 years of experience? When I moved from graphics into Mac software, some people seemed to think I must have done it under duress of some kind, and that I’d be desperate to return to my previous stomping ground. Some people don’t seem to get that it’s possible to be interested / enthused about more than one subject area, and that over time this already complex set can change. Why is that?

Well, people like attaching simplifying labels to things. It’s convenient for sure, but people are a naturally complex collections of skills, interests and desires that you really can’t pigeon-hole, even if you really want to to avoid dealing with the real depth and constantly shifting complexity that’s actually there, and how actually this should be considered to be a benefit, rather than an niggling inconvenience to your categorisation system.

There’s also a perception that you’re ‘throwing away’ all your experience when you jump into a different subject area or industry. This is just plain wrong too – you never lose any of the knowledge you’ve built up (besides poor memory perhaps but rusty knowledge is polished up quite easily), but the differential of this knowledge – how much you’re learning right now – decreases hugely over time. After 5 years in a similar subject area, you’re probably not learning very much anymore; maybe you’re considered an ‘expert’ now, which maybe some people like from a prestige point of view, but to me the lost opportunity of learning far more in a new subject area outweighs that considerably.

I think the tragic thing is that a lot of people end up pigeon-holing themselves because it feels like that’s expected of them, because society appears to reward only the specialists. I can’t help thinking that a lot of mid-life crises are attributable to this dynamic.

Business Personal

On being acquired

A lot of you will already know, but SourceTree, a Mac client for Git and Mercurial I created over the last 18 months, has just been acquired by Atlassian. There’s a press release, articles on TechCrunch and VentureBeat, and an official FAQ on the SourceTree site. But this is my personal blog, and I’ve had a few requests for a personal angle on this, so here you go.

I said in a previous post that in my experience, the best opportunities often come along when you’re not looking for them, and that was certainly the case here. I wasn’t even thinking about looking for acquisition opportunities for SourceTree – sure, the idea had crossed my mind as something I might want to consider eventually, but it certainly wasn’t an active line of thought this early in the product lifecycle. SourceTree had grown to become a viable business for me, and I was very much enjoying the process of just creating a software product that I used every day myself too.

So, when the Atlassian opportunity came up, I wasn’t at all prepared for it, and I had to make some decisions. I was enjoying being master of my own destiny, and was managing just fine – so my initial knee-jerk reaction was to be very cautious. However, the more I thought about it, and the more I learned about Atlassian, the more I realised what a huge opportunity I’d be turning down, both personally and for SourceTree, if I said no.

Any acquisition kicks off with a financial offer (don’t expect details, they won’t be forthcoming ;)), but that’s far from the whole story. In my case I didn’t have any pressing need to sell, and I’ve learned from experience that being happy about what you do is extremely important. I also have a strong attachment to the products I create – that’s why I stayed with Ogre for 10 years and it was/is still a wrench to leave – and that’s the case with SourceTree too; not to mention that I’m a daily user of it myself. So if I was going to sell, it had to be to the right company who would look after it just as well, or better, than I did.

Luckily for me, I discovered that Atlassian was about as perfect a fit for SourceTree as I could have asked for. Atlassian lives and breathes developer tools – that’s their entire product focus, which in itself is a good start. They’re investing heavily in DVCS tools – hence the 2010 Bitbucket acquisition and its recent enhancement to handle Git as well as Mercurial (which of course SourceTree does too) – again spot-on on the compatibility chart. I learned, particularly when I visited their HQ in Sydney, that everyone at Atlassian really ‘gets’ developers (well, most of them are developers after all), and care a lot about giving them good products. Development tools permeate the entire company – the CEO is a regular user of SourceTree, and people in marketing understand when you talk about version control. Even though they’re quite a big company now, it retains a startup feel. Then there’s their corporate values, which are very much in evidence when you talk to people there – things like “no bullshit” and “don’t f**k the customer”. And it’s not just on the wall, it’s really how people in the company make decisions. These are the kind of people I can relate to, and definitely the kind of people who can add a lot to the future of SourceTree.

Another thing I found reassuring is that at no time was there any question that Atlassian would want to railroad developers into their own tools at the expense of others. Clearly Atlassian already owns Bitbucket, and SourceTree supports Bitbucket, GitHub and Kiln already. It was made abundantly clear to me that no-one at Atlassian took the view that restricting developer choice to favour Atlassian tools was a good idea. Their ideology is to make developer’s lives better by giving them choice, and of course they’re going to want to offer good Atlassian options in there, but if a developer wants to use an alternative, no-one is going to stop them.  The view I got from everyone was that giving developers a positive experience that reflects well on Atlassian, including giving them their own choice of integration, is far more valuable than artificially chaining them in. Obviously, I concur.

My final reason was that while I really, really enjoyed creating and supporting SourceTree myself, the workload is quite high, and was increasing. It’s not a continuous death-march, but the availability requirements are very high – since I was developer, webmaster, sales, customer support and everything else all rolled into one, taking a day off was basically impossible. Making sure the website was still up, and making sure customers got a quick response to support calls, was a 24/7 responsibility. After a while, that gets tiring, even just checking on things all the time means you never have ‘proper’ downtime. A big advantage of joining Atlassian is that I get some extra backup. That’s good for my health & mental wellbeing, and I’m sure that will be good for SourceTree too long-term. I really didn’t want to start resenting SourceTree for preventing me having a proper holiday occasionally ;)

So based on all these factors, I decided that the future for both myself and SourceTree would be better within Atlassian than continuing alone. I learned a lot along the way to this acquisition – dotting all the i’s and crossing all the t’s turned out to be more time consuming and stressful than I expected, so I wouldn’t say it’s a process for the faint hearted, but if you’re as lucky as I was to be approached by the right company, it can lead to a really great outcome.

I’m still fully committed to developing SourceTree, like I was before, but now it has a more robust support structure around it. Taking things to the next level, both in terms of user base and features, is so much more practical now within Atlassian. I’m very confident that they’re the right company to take SourceTree forward – our thinking is very similar, I respect their values a great deal, and the people are great. My decision was a lot easier than it might have otherwise been because of this!

Development Personal Tech

My Version Control System History

I was thinking the other day about how many version control systems I’ve made my way through over the years of being a professional developer, and I figured it would be fun to put it in graph form. Of course, this is entirely from memory and gives the illusion of being more empirical than it actually is, but hey, everyone loves graphs, right?

Yes, I really didn’t use any source control back in 1994, barring backing up to 3.5″ floppies. ‘Custom’ refers to a system I wrote myself in 1995 to do version control on a mainframe system I was working on at the time; really it wasn’t a proper VCS system as you would think of it now, but it did the very basics to make sure we knew what was being developed and deployed at any point in time. You could think of it as a VCS system that only had history at the tags ;)

As you can also see, Visual SourceSafe was the first packaged system I used – you can pity me now – as more of my work started to involve PCs instead of ‘big iron’. At that stage, the concept of ‘checking out’ and ‘checking in’ was normal. Oh, how naïve we were.

I discovered CVS at the turn of the millennium, at the same time I started Ogre. The ability to work without checking anything out was a revelation! I went on to use that on all my projects for quite a few years, at least until I considered Subversion stable enough to base production work on. Even then, I only moved new projects to Subversion mostly (Ogre eventually moved in 2008 – was it really that recent?), and kept CVS for many existing projects.

I first encountered Distributed Version Control at a Git presentation at the Summer of Code mentor summit at Google HQ in late 2007. I have to admit, I didn’t really ‘get it’ at first – it just seemed to make things more complicated and most of those promoting it were mostly die-hard Linux fans and wide-eyed early adopters that didn’t make you want to put your production code on it yet. There was also no GitHub at this time so the benefits weren’t really very visible. It took me until 2009 to really start evaluating DVCS in earnest, and decided that I liked Mercurial best, due to its equal treatment of all platforms, good diagnostic reporting, easy extensibility and intuitive interface that gave power without over-complexity. Today, all my own projects are in Mercurial, but I use Git a lot too, not least because I support it in SourceTree, and get involved in other projects which use Git.

There are a couple of randoms thrown in there – I used Alienbrain for a few months on a client project and totally hated it (think Visual SourceSafe with a few bells – it may be useful for artists but for coders it was horrid), and I did play with Bazaar for a bit but didn’t really warm to it, even though it had some quite nice ideas.

So there you are – my VCS experience over ~17 years in professional development. It kinda makes me want to do a language / platform graph now :)

Business Cocoa Development Objective C Personal Tech

Introducing: SourceTree

I’m pleased to announce that I’m finally ready to make my first fully-fledged commercial Mac OS X application available to the world!

SourceTree is a user-friendly Mac OS X front-end for Mercurial and Git, the two most popular distributed version control systems used today. The goal was to create a single tool which could deal with both systems efficiently, and to give a developer quick and intuitive access to the things (s)he needs to just get on with building software.

I thought I’d answer a few background questions on this that I get asked on occasion:

Why Mercurial AND Git?

Other apps tend to concentrate on just one version control system, so why am I supporting two? Well, as a developer I’m regularly coming across projects from both sides of the fence, and in practice I find I need to use both fairly regularly. I personally chose Mercurial for my own projects (and discussed why here), but I still use Git when dealing with other projects, and spend a fair amount of time hopping between the two. It struck me that even though they have their differences, they are both based on the same distributed principles, so having to use two separate tools was just unnecessary. I wanted a single tool which provided a common interface where that made sense, while still exposing the things they do differently where that was useful too. SourceTree 1.0 is my first attempt at that.

Why only Mac OS X?

There were actually multiple reasons for this choice:

  1. I wanted to learn Objective-C and Cocoa on a real project
  2. I know from experience that designing for multiple platforms can be a distraction, with more time spent on compatibility issues, and less on functionality – and that’s before you even consider the compromises  you have to make, particularly on UI conventions which are far from uniform across platforms. I’ve been a multi-platform developer for more than 10 years, and for a change I just wanted to focus on the end user results and nothing else. I’m aware that schedules slip very easily when you overcomplicate, and I’m already supporting multiple DVCS systems (something I consider to be an important feature point), so I deliberately chose to keep this element simple.
  3. Mac OS X has become my own platform of choice for most things now. The combination of stability, user-friendliness, Unix underpinnings and well designed hardware match my current needs perfectly. I’m done with the ‘some assembly required’ PCs that I loved tinkering with over the past 15 years

What about Subversion?

A few people have asked me if I plan to add Subversion support too. I actually did intend to originally, until I realised how much time it was going to take to just do a decent job on Mercurial and Git. Within the time constraints, I focussed on the subject areas that I felt I could contribute most to – there are already quite a few Subversion tools out there for Mac OS X, but Mercurial and Git are much less well served, so that’s where I focussed my efforts.

I still have Subversion support tentatively on my work plan, but it’s not top of the list. I think it’s better to do your most important features well before diversifying. Plus, there are problems with Subversion – it’s very, very slow compared to Mercurial and Git, so to match the performance in SourceTree of things like the interactive searches and dynamic refreshing / log population I’d probably have to do a ton of extra caching just so the user wasn’t sat tapping their fingers.

Edit: I made my decision on this: I don’t plan to support local Subversion, but to support operating with Subversion servers with Mercurial and Git locally via hgsvn and git-svn.

Why didn’t you make it open source?

Sorry folks, while I love contributing to open source (I’ve done a bit on SourceTree too, sending a patch back to BWToolkit), making it work as a business is very hard indeed. I half-killed myself trying to combine being an open source project leader and doing other commercial activities at the same time, so now I’m trying a more traditional approach. One thing I learned in the last few years is that there are some sectors & application types where being an open source maintainer is very compatible with also running a business based on that project, and there are others where you can really only do one or the other simultaneously without flaming out. Sucks, but there it is ;)

What’s Next?

I have a public, official roadmap for SourceTree and encourage users to suggest things they think should be on there, via the support system. I learned from running an open source project for 10 years that being open about your plans can be a big benefit – users like to know where things are likely to be going, and often have better ideas than the developer on what could do with a bit more spit and polish. They can also tell you what’s important to them, which is crucial for prioritising – as developers we tend to get carried away with things we want to work on, but in the end, it’s scratching the customer’s itch that matters most.

And while I’m really quite proud of SourceTree 1.0, there are plenty of features I’d like to continue to add, and definitely more room for some totally unnecessary beautification which I didn’t have time for in the first release. Hey, this is OS X ;)

SourceTree is available now on a 21-day trial license. Go get it already :)