Category Archives: Tech

Development Tech Windows

Friends don’t let friends use ClickOnce

May 12, 2013 – 4:03 pm

I’ve had a fair amount of experience with Windows-based installers in the past, including non-Microsoft Installer based systems like NSIS and the open source WiX, but most of the time I’d been working with one-off installers for native code projects, like the Ogre3D SDKs. I pride myself in not pre-judging the best toolset to use for any given problem – which is why I switch languages a lot – so when I came to write SourceTree for Windows, which is based on .Net, I checked out the landscape again.

I discovered that ClickOnce was Microsoft’s recommended approach to .Net application deployment , and had the advantage of including both the installer tech and an auto-update system, together with Visual Studio integration, a simple deployment model and so on. I’d seen it used on a couple of other products, and during testing both internally and with our batch of beta testers all seemed to be well, so that’s what we went with for the release.

Hindsight is a wonderful thing, and it turns out that ClickOnce has a number of problems which only really show up when you use it at scale, by which time it’s a bit too late and you can find yourself with a lot of support requests. Here are some of the issues:

  1. No support for proxy servers
    It turns out that ClickOnce just doesn’t work at all with secure proxy servers which can’t authenticate automatically. It also turns out that a lot of people have this kind of setup. There’s a (nasty) workaround for the auto-update system, but this can’t work for the first install. Microsoft have known about this problem for some years, but have decided not to bother fixing it.
  2. Packaging format is just a folder of many files
    ClickOnce puts all of its files in a folder per version and just tacks ‘.deploy’ onto the end of each. Simple, but it has many problems:

    • Some corporate firewalls will detect that some of those files are executable binaries, and will flag them as dangerous. Some will totally block the file making the install/update fail with a download error, some will ‘sanitise’ the file and send that on, making the install fail later with an ambiguous hash mismatch error.
    • If you use a CDN to distribute your installers, distribution of many files is very difficult to make atomic, potentially leading to temporarily incomplete results being visible
    • Having the download (first install and update) as a multi-file affair makes failures due to interrupted connections etc much more problematic, vs just resuming the download of a single package.
  3. You can’t separate the update metadata from the update itself
    That is to say, ClickOnce forces you to use a single URL for both the metadata where it finds that there is an update, and where it gets the actual update from (the installation files are in a subfolder). This might seem sensible, but in practice it’s very limiting. If they’re separate, you can point the metadata URL at a location which you can update in-place and which will never change, and you can put the update files themselves wherever you like (a CDN usually). ClickOnce forces you to put both in the same URL, which is limiting when it comes to reorganisation, and also it can mean you have to put the metadata on the CDN too, and CDNs don’t often like files that update in-place. It’s just an awkward and inflexible system.
  4. Random weird sh*t in a small percentage of cases
    We’ve found over the last couple of months that occasionally a user’s machine refuses to install or update despite the ClickOnce installer being fine for everyone else, and in circumstances that were not explainable by the other issues listed above. It’s not that common, but even if it’s only 0.5% of cases the numbers were starting to add up. We often managed to work around the problems manually, but it’s just not something you want to spend your or your customer’s time on.
  5. Other niggling limitations customers don’t like
    None of these are deal-breakers in their own right, but combined with everything else they’re worth mentioning:

    • You can’t choose where to install the product – ClickOnce chooses an esoteric location for you
    • ClickOnce’s reliance on the logged in user means you can’t install once for everyone on a PC, or distribute a bulk install as admin
    • No offline installer – while you can zip up the deployment it didn’t work for everyone

So, all these things have been building up , tempered with workarounds but nevertheless a constant thorn in our side. In the end, we had to make the call that ClickOnce just wasn’t salvageable as a deployment tech to use at scale. Instead we’ve moved to a standard Microsoft Installer based tech stack, using Advanced Installer to build the installer and the update system. I really liked Advanced Installer because not only does it address all the issues above, it saved me a ton of time over building something myself with WiX (and having to solve the prerequisites bootstrapper and auto-update system), and was also just complex enough to do what I needed; it’s a pro tool but doesn’t require 3 months training to master it, you can pick it all up in a few days.

I wrote this blog post because it’s what I wish I’d been able to read before I decided to use ClickOnce. It’s one of those technologies that looks fine on the surface and in limited testing but then reveals a bunch of horrid edge cases and random unreliability issues  in the wild, just when you don’t need it.

Tags , , , | Permalink | Comments (8)
Personal Productivity Random Tech

Evading the now

March 24, 2013 – 10:26 am

Ever since I notionally transitioned into adulthood, I’ve always been interested in current events. Not just keeping up with the latest technological developments either, news in general is something I like to keep tabs on. Years ago, you basically had two sources: the fairly superficial summaries from TV news and tabloids, or the more in-depth coverage from broadsheet special reports and dedicated periodicals. Generally speaking you got the superficial once or twice a day, and something more probing every week/month.

The Internet changed everything, including the news. I still believe that change is for the better – I remember what it was like to try to find specialist technical information in the years before Google, and I can’t believe how I managed – but I also think that lots of us don’t manage this tsunami of information availability very well, particular when it comes to its frequency. I think the constant stream of information at all levels of detail has overwhelmed some of us to the extent that we don’t know how to stop looking at it, and this has some knock-on effects for creativity and independent thought.

I started thinking about this because I recently took a 2 week holiday, and for the first time in many, many years I took the opportunity to be completely unavailable for the duration. Anyone who has been self-employed, or simply personally committed to a project to the level that they don’t know how to let go, knows what a huge deal this is. Frankly, I badly needed the proper down time and so did my wife (unfortunately the net effect has actually been to push extra stress on to the surrounding weeks, but at least we could relax for a while). It turned out that the place we stayed had almost non-existent Internet access anyway, so my choice was enforced.

So for 2 weeks we had no Internet, TV, radio, papers, nothing – the world outside may as well have not existed. We just had a bunch of pre-loaded Kindles, iOS devices and notebooks – no twitch-checking Twitter, no news, no daily RSS. At first, it felt isolating. Then, it felt liberating. I found myself feeling much more creative, and being able to think about the ‘long game’ a lot more clearly. It wasn’t that I didn’t care what was going on in the world, or that I had any less of a need overall to keep up with technical developments etc. The point was that I didn’t have to think about it so often – catching up with it in a week would be absolutely fine. Or two weeks. Anything ‘important’ that I missed would get flagged up again anyway, and in the meantime, I found there was much more free space in my daily consciousness, not only for relaxation and enjoyment, but for creativity and strategic thinking too.

Looking back it should be obvious. More quiet time to think has to be better, right? There was also a study recently saying that kids should be allowed to get bored because it makes them more creative, and I bet it works for adults too. I’ve always been a bit contemplative anyway, quite content sitting quietly and just thinking. But without realising, I’d sidelined that almost entirely from my routine – extroverts always had this ‘work hard, play hard’ mantra they lived by, but without realising I’d adhered to an equivalent for introverts – ‘work hard, consume new information obsessively in free time hard’ perhaps. Before the Internet, and more specifically before the ‘velocity’ of the Internet increased via Twitter etc, there were lower frequency rhythms in the way new or topical information became available that naturally paced my consumption and the depth/length of contemplation on each, but once the limits were off I slowly became like an ex-hunger striker at an all you can eat buffet. I didn’t even realise it had happened until I went cold turkey.

So, what next? Obviously I’m not going to stop using Twitter, or reading tech news. But what I am going to do is deliberately unplug regularly for periods of time in future. I think a couple of days at a time is the minimum required to reach the kind of ‘quiet brain’ that seemed to me to be much more cogent and creative than the one I live with most days. A weekend a month perhaps, plus at least one full week a year. It’s a bit like clearing all the crap off your desk once in a while – which come to think of it I should also do more often ;)

Tags , , | Permalink | Comments (3)
Development Health Personal Tech

104 keys: who needs them anyway?

July 22, 2012 – 4:42 pm

I fired up my desktop Windows PC for the first time in a while recently, and the first thing I realised is that I absolutely hated the keyboard. This was nothing to do with the slight differences between the PC and Mac keyboard layouts, the latter of which I’ve become more accustomed over the last couple of years, nor was it about whether the keys were mechanical or scissor switched, or any such nuance.

No, it was solely this: the damn thing sticks out way too far on the right hand side if you’re a right-handed mouse user.

Like most heavy computer users, I’ve had various bouts of RSI over the years. The most basic case was resolved by learning to touch-type rather than the spider-typing that actually most programming languages encourage you to self-teach if you don’t know any better, given that all the useful symbols are around the edge of the keyboard. Later on, I found that low-profile keyboards with low travel distances & pressure resistance reduced general fatigue-related issues I started to get with age. But there was one issue which never went away – aches caused by mouse use. The main issue for me was not the mouse itself, but that because of the asymmetrical nature of the standard PC keyboard, with the cursor / navigation block and numpad sticking out to the right, your mouse has to be placed quite far out to the right, meaning that when using the mouse your body is skewed out of symmetry, meaning you’re straining one side of your body more than the other. This gives me shoulder & back ache, and also the angle means the unnatural wrist rotation of most mice is even more exaggerated, causing issues there too. I try to use keyboard shortcuts as much as possible, but you can’t avoid reaching for the mouse sometimes.

Now, I’ve been using a MacBook Pro pretty much exclusively for the past 2 and a bit years, and all those aches have gone away, because the trackpad is right there in the centre and everything remains aligned. I love the Mac laptop trackpads – they’re large, expressive, multi-touch, ergonomic and just a joy to use, to the extent that I can’t use any other trackpad without being struck by how awful they feel. I’ve tried a few on Windows laptops over the last few years and my response is always really negative. And this is a desktop in any case, they’re quite hard to find outside laptops even if I wanted one. So in practice, a PC-based trackpad wasn’t going to cut it, the mouse really had to stay.

Secondly, since I’ve been using a laptop keyboard for so long, I don’t feel I even need all those separate blocks of keys any more. Sure, I used to use them all the time;  I’d used 102/104 keyboards since 1991 and my muscle memory had got me used to jumping over to the cursor keys, Home/End, and occasionally the number pad. But in the last 2 years, that tendency has been completely eradicated from me – I now reach for the arrow keys in the bottom-right of the keyboard, and use combos like Fn-Left-Arrow rather than needing a dedicated Home key without even thinking about it.

So here I was, with a keyboard that was throwing off my ergonomic balance due to a couple of blocks of keys I didn’t need anymore anyway. I could have just sawn the keyboard in half I suppose, but I looked online and ended up grabbing a surprisingly cheap ‘compact’ keyboard that just gets rid of all that. You can see the difference in the image below:

To me, this feels a lot more natural; the angle of my right arm is far less acute, and that puts less pressure on my shoulder and wrist as well as not requiring me to reach as far when I move back and forth from the keyboard to the mouse.

The keyboard in question is not great quality, (I would have bought this one instead if it wasn’t for the fact that it only came in American layout with the horrible single-row Return key) so I’m going to have to see how that pans out. The range of choices for keyboards like this seems very limited, with a few options like this one at the budget end (probably designed for server rooms methinks), and a few crazy expensive options at the ‘specialist ergonomic’ end, which I wasn’t ready to pay for (I don’t expect to be using this machine all the time). If you know of any others for future reference, please let me know in the comments.

A final thought: over the years I’ve become more aware of how we seem ready to accept working in configurations that are actually quite bad for us physically, due to convention and familiarity. Almost 10 years ago a colleague of mine bought a Fingerworks TouchStream LP, a weird multi-touch surface that acted as both a keyboard and a mouse at once, with gesture support, and which at the time I thought was total insanity (and I found it impossible to use his PC!). In hindsight, I recognise the genius in that product, and can see why Apple bought the company – who of course ended up being at least partly responsible for how good Apple’s touch-based tech feels today.

Tags , , , | Permalink | Comments (19)
OGRE Tech Web

Apache to Nginx & PHP-FPM : part 1

May 9, 2012 – 3:42 pm

Background
The Ogre3D website has been running on a dedicated server for about 7 years now; this is relatively expensive, but when we moved away from the shared hosting that Sourceforge generously provided, but which we had outgrown, our initial foray with a VPS (at the time lighttpd on Linode) proved inadequate for our needs, so after a month of futile tuning we gave up and went fully dedicated.

Time has moved on of course, and virtualisation technology is considerably better than it was in 2005. I’d intended to try again soon anyway to reduce Ogre’s overheads but our Adsense revenue was still covering the cost and I hadn’t got around to it yet. Then suddenly, Google pulled our ads after a mistaken (I believe automated) conclusion that we were hosting copyrighted material – a few users had posed test binaries of their own work on MediaFire and similar ‘red flag’ download sites – and all of a sudden we were leaking money. The misunderstanding was sorted out with Google within a few days, but even so it illustrated that we should probably look to move to a cheaper solution if we can so we have less exposure.

The Ogre site’s main issue with performance is Apache’s memory usage under load, so given a VPS is more constrained I wanted to address that. Enter Nginx, stage right.

read more »

Tags , , , , | Permalink | Comments (8)
Personal Tech Web

Blog revamp

April 29, 2012 – 3:57 pm

As I promised, I’ve given this blog a bit of an overhaul in anticipation of posting more often again. For those who are interested, here’s a run-down of the updates:

  1. New responsive design
    Responsive design is all the rage these days; in summary, it’s all about making your site adapt fluidly to the reading environment so it looks good on a variety of devices, even resizing images so they always fit. Try resizing your browser window, and you’ll see that the layout adapts, dropping the sidebar when it gets too thin and so on. I’d love to take credit for it, but monkeying about with CSS is one of my least favourite things in the world so I just used/tweaked an existing theme which took my fancy. Clearly, I’m in a minimalist phase right now.
  2. Commenting via Disqus
    Previously I’d used the standard WordPress commenting system plus a reCAPTCHA plugin to cut down on spam. That worked well enough, but Disqus has some advantages, chief of which is that you can comment using existing accounts such as Twitter, Google, OpenID and Facebook, which is much faster and frictionless. Also if you have a Disqus account, it collates all your comments across all websites so you can reference them more easily – I’ve found this useful myself in the past if I’ve commented on a post I found on Twitter and then forgot where it was. Importing all the blog’s existing comments to Disqus was easy, and allegedly all comments in future will be stored in both places (edit: confirmed, this is working), so you don’t have to worry about being exposed to external data loss, you always have all the comments in your own database too.
  3. Replaced many scattered plugins with Jetpack
    I used to use a bunch of different plugins for twitter feeds, subscriptions and social sharing buttons, but then I found Jetpack, which packages a bunch of them in one plugin.  Seems to work well, and it beats having to upgrade a bunch of separate plugins.

So, I hope you like the new design. Apologies for any lingering issues, I’m sure there will be some tweaks to do in the coming days but generally I think it’s a big improvement.

Tags , , , , | Permalink | Comments (7)
Personal Tech

Google+

July 4, 2011 – 5:10 pm

I left Facebook about a year ago and have been using Twitter as my primary social tool ever since. At the heart of this decision were my main gripes with Facebook:

  1. Facebook misrepresents relationships
    It’s clear that Facebook was designed by a young person with borderline Aspergers. Relationships are black and white, you’re either a Friend or you’re not, and they’re symmetrical – information has to flow both ways. The real world doesn’t work like that, I have real friends, family, casual acquaintances, people I like to keep up to date on but who I’ve never met, and people who like to follow me but who I don’t know. I interact with every one of these groups of people differently, and very often in asymmetrical ways. The concept of a ‘Friend’ who I’d share private information with online is completely unrealistic. I know Facebook bolted on ‘lists’ but everything is totally hobbled by this misguided overarching ‘Friend’ concept.
  2. Facebook’s signal-to-noise ratio is unmanageable
    Whether it’s automated posts from games & other poorly considered apps, or just people pouring every trivial little element of their life into it, I was drowning in trivia in a few months to the extent that it was a chore to keep up to date. This is a function of the people you’re connected with of course, but here’s the problem – because of the way Facebook simplifies relationships, you can’t really do anything about it without offending someome, because:
  3. Facebook creates social awkwardness
    Because Facebook only recognises one type of connection as pointed out in 1, it makes managing your information stream impossible without offending someone. For example, I may very well have real friends / family who I want to connect with, but who I don’t necessarily want to listen to 24/7 because they have .. ahem.. ‘very poor communication filters’. That doesn’t mean I don’t like them, we just communicate very differently. Sure, I might want to dip into their updates every so often, but I don’t want to have to wade waist-deep through their posts every time I go online to find the gems from people who are better at filtering themselves.
    On Twitter, I just unfollow people who I don’t feel like listening to every day, and trust others to RT things that are good (I can always re-follow later). They don’t get told, and generally it’s not considered offensive. Conversely unfriending someone on Facebook is equivalent to saying you never want to speak to them again – it was literally easier just to close my entire Facebook account than to deal with it any other way. Sure you can create a Facebook group, but since groups are visible to all, creating one called ‘People I like in person but who are kind of annoying online’ isn’t going to avoid the problem.

So I went with Twitter because it basically sidesteps all these issues by being public (lack of ‘pretend’ privacy leaves no-one under any illusions about appropriateness), asymmetrical (I can follow someone without them following me and vice versa), and easily managed / filtered without offence. I largely had given up on a ‘rich’ social network that worked the way I wanted, because everyone else seemed to swear by Facebook’s way, which I hated.

Then, last week Google came out with Google+, their latest answer to the problem. Invitations are hard to come by, they’re still in limited testing, but I managed to sneak in before they closed a loophole, ironically via friends on Twitter. I didn’t expect to like it – I expected another Facebook – but I was wrong. Key to what makes Google+ different is that it doesn’t use the concept of ‘Friend’, but instead uses a core concept of ‘Circles’. Google have done what Facebook have refused to do, and have recognised that relationships aren’t binary, aren’t always symmetrical, and are all different. Circles can be used to filter the things that you post into groups of people who can see it, but just as importantly to filter incoming posts too. Most importantly, the person can’t see what Circle you’ve put them in! So if you like the person in real life, but find their online posts a bit vapid, you can put them in a circle that you don’t have to read as often, and they don’t have to know. No-one has to get offended, and you get to keep your stream sane. Perfect.

So far, I like it – it’s taken the best bits of Twitter (control without offence) and Facebook (rich media platform) and slapped them together. Of course, there’s the question of whether people wedded to Facebook will switch or not, although for me that’s a moot point since I already ditched it, so any people on it that do come across to G+ are a bonus, but I lose nothing if they don’t. Twitter users seem to be quite eager to adopt it, probably because they’re natural early-adopters and appreciate the Twitter-like circle setup for the same reasons I do.

And it’s for that reason that I think Twitter has most to fear from Google+. Facebook will probably trundle on with all the people who don’t particularly care about managing their streams in an optimal way and don’t feel a need to switch. Twitter though is used by a lot of people (like me) who are a lot more picky about this sort of thing, and those are exactly the people for whom Google+ will resonate the most. Right now, the main limiting factor is that there are no native tools for Google+, and using it on a phone or iPad is still a bit clunky compared to the kind of optimised native tools you can use for Twitter. But that’s coming I’m sure.

I haven’t even talked about the group video chat, or the really nice photo viewer, how Google+ notifications appear when you’re on any Google site (search, GMail, Reader – a huge advantage for adoption), or how they’re iterating really fast on this. In all, I think it has great promise to be a social network that doesn’t annoy me – yet. One of the big challenges as they develop will be how they handle bot-posting from the inevitable games and other applications once the API is released (I hate it when people hook up everything under the sun to auto-post to social networks, and I unfollow them on Twitter even if they’re friends). We’ll see.

Tags , , , | Permalink | Comments (5)
Business Development Tech

Why ‘software engineering’ is a misnomer

June 16, 2011 – 6:07 pm

These days I’m a free agent, and I’m lucky enough to be able to choose what projects I work on, but in a past life, I was what I suppose is properly referred to as an ‘enterprise software developer’. Yes, I once functioned in an environment where terms like ‘mission-critical’, ‘project life-cycle’, ‘stakeholders’ and ‘change management’ came up quite a lot. I’m grateful for the experience I gained over 12 years of doing that, but I’m also very glad to be free of it now.

One term which was bandied around a lot in this environment was ‘software engineering‘. The implication of this term, and the expectation of many people using it, is that designing and creating software is very much like physical engineering disciplines, such as erecting a bridge or building a block of flats. When a software project starts to go off the rails, and fingers are starting to be pointed, a common question is why large physical engineering projects can come in on time an budget, when most software projects beyond a certain size don’t? Surely if the people who create software claim to be ‘engineers’, they should be able to adhere to the same standards of other engineering professions?

At face value it sounds like a fair argument, and I’ll be the first one to admit that software engineers are very often not as rigorous as others bearing the title. But there’s a good reason – the belief that software projects are akin to building bridges is completely misplaced, and it is a belief which too often leads people to think that software projects should be run using techniques such as fixed prices, up-front design, and waterfall life-cycles. This is despite decades of case studies indicating that such techniques rarely work, and are more often a collective delusion adopted because the reality – that you’re never going to be able to predict a large software project accurately up-front – is too terrifying to be contemplated. The term ‘software engineering’, while indicating some of the responsibilities of such a person, is an incomplete definition of what building software actually entails, and just reinforces these misconceptions.

So, why is making software so different to building a bridge?

  1. Software is intangible.
    You can see and touch a bridge, it has a physical presence you can measure and its purpose is absolutely obvious to any observer, as are its success and failure conditions (getting people across a gap without falling down). Even though there are parts of the bridge the user will never see, their purpose can be easily unambiguously defined – carrying utility cables/pipes, providing maintenance access, etc. Software is intangible, and even the part that is seen by those defining its purpose is often hard to evaluate until it can be seen and used directly, never mind the internal working processes. Evaluating and defining software ahead of time is like trying to describe a dream seen through a keyhole – incomplete and everyone has a slightly different impression, and regardless of how much you try to write it down, you can’t capture or communicate its essence completely to others, or guarantee that they interpreted it the same way.
    People are just naturally very bad at defining and evaluating intangible things. There’s no point pretending they can with 300-page requirements documents, you’re just wasting everyone’s time – accept that people won’t know what they like until they can play with it – iterate!
  2. Interaction complexity
    There’s only a small number of ways to use a bridge, and they’re each quite straight forward. Yes, there’s much complexity to the structure of the bridge itself, but that really doesn’t matter to a person driving or walking across it – that doesn’t need input or feedback from people except for whether they like the look of it.
    Software is basically defined by complex interaction, which is often context- and data-sensitive. The number of variations mean that even if every one is bug free (yeah, right), there’s the question of whether it’s solving the correct problem, which is a function of people and their interpretation / opinion. While building projects can be complex, those complexities are most often a function of the engineering challenge, whilst most complexity in a software project is a function of the involvement of people, which are far more difficult to pin down.
  3. Change is a constant factor
    Change happens everywhere of course – maybe the bridge builder discovers that the ground isn’t as firm as they thought, or suddenly someone wants to make an extra deck available for trains. Big disruption, and an effect on time and budget – all understandable. The problem when it comes to software is that the perception of change is very different. If suddenly you have to sink massive extra foundations or double the amount of steel you need to use, people understand the magnitude of that intuitively and accept the effect on the schedule. Changes in a software project tend to be perceived as ‘just a little tweak’ by those requesting it, which tends to mean tolerance for schedule impact is lower, and indeed the potential for interaction between changes in unexpected ways is often overlooked. Now, I’m not for a second advocating that change should not be allowed, but often the mechanisms that have been established for managing a project – up-front estimation, fixed feature sets and so on – prevent the efficient integration of change, or at least obscure its impact to a point where things become critical. It’s important to appreciate that change is more common and expected in software projects, by nature of the issues raised in the previous points, and therefore setting your expectations based on a discipline where change is less common is not a smart move.
  4. Lack of hard limits
    Building  a bridge involves respecting some hard, unchallengeable limits. As a wise man once said “You canna change the laws of physics, Cap’n!”. When you’re building software, however, there are very rarely any unbreakable limits, and even the soft limits that there are tend to be invalidated very regularly as new technology comes on stream. No-one can realistically stand up and say ‘this is impossible’ when faced with a request, because almost nothing is impossible. Hard limits in physical projects make all stakeholders ‘get real’ – you can’t use more land than you have, you can’t build a skyscraper out of paper, there are non-negotiable rules for safe load-bearing and so on. Barring some really crazy and ground-breaking structures (which BTW, rarely come in on time & budget), these limits ground everyone involved in the project to realistic expectations. In a software project, reining in those feature requests, or those ideas that the implementors have for a new tech they could use, is much more abstract affair, and everything is up for debate.

All these things put together mean that if you think you can use the same kinds of techniques to run a software project as you would use for a building project, you’re kidding yourself. They’re so very different in fundamental ways, it’s like trying to catch a shark with a shrimping net (because it’s just a fish, right?).

So please, stop with these delusions:

  1. that estimating the cost of any non-trivial software project (that isn’t just a carbon copy of something else) is any more sophisticated than mildly informed guesswork.
  2. that it’s reasonable to think that a fixed-price can be realistic for any software project longer than a couple of weeks
  3. that ‘change control’ is the about exceptions, rather than the norm

Of course, agile approaches attack these particular delusions rather well. But try to get those adopted in very large organisations where the ‘software is like a bridge’ mentality still thrives :)

Tags , | Permalink | Comments (8)
Development Personal Tech

My Version Control System History

February 19, 2011 – 1:05 pm

I was thinking the other day about how many version control systems I’ve made my way through over the years of being a professional developer, and I figured it would be fun to put it in graph form. Of course, this is entirely from memory and gives the illusion of being more empirical than it actually is, but hey, everyone loves graphs, right?

Yes, I really didn’t use any source control back in 1994, barring backing up to 3.5″ floppies. ‘Custom’ refers to a system I wrote myself in 1995 to do version control on a mainframe system I was working on at the time; really it wasn’t a proper VCS system as you would think of it now, but it did the very basics to make sure we knew what was being developed and deployed at any point in time. You could think of it as a VCS system that only had history at the tags ;)

As you can also see, Visual SourceSafe was the first packaged system I used – you can pity me now – as more of my work started to involve PCs instead of ‘big iron’. At that stage, the concept of ‘checking out’ and ‘checking in’ was normal. Oh, how naïve we were.

I discovered CVS at the turn of the millennium, at the same time I started Ogre. The ability to work without checking anything out was a revelation! I went on to use that on all my projects for quite a few years, at least until I considered Subversion stable enough to base production work on. Even then, I only moved new projects to Subversion mostly (Ogre eventually moved in 2008 – was it really that recent?), and kept CVS for many existing projects.

I first encountered Distributed Version Control at a Git presentation at the Summer of Code mentor summit at Google HQ in late 2007. I have to admit, I didn’t really ‘get it’ at first – it just seemed to make things more complicated and most of those promoting it were mostly die-hard Linux fans and wide-eyed early adopters that didn’t make you want to put your production code on it yet. There was also no GitHub at this time so the benefits weren’t really very visible. It took me until 2009 to really start evaluating DVCS in earnest, and decided that I liked Mercurial best, due to its equal treatment of all platforms, good diagnostic reporting, easy extensibility and intuitive interface that gave power without over-complexity. Today, all my own projects are in Mercurial, but I use Git a lot too, not least because I support it in SourceTree, and get involved in other projects which use Git.

There are a couple of randoms thrown in there – I used Alienbrain for a few months on a client project and totally hated it (think Visual SourceSafe with a few bells – it may be useful for artists but for coders it was horrid), and I did play with Bazaar for a bit but didn’t really warm to it, even though it had some quite nice ideas.

So there you are – my VCS experience over ~17 years in professional development. It kinda makes me want to do a language / platform graph now :)

Tags , , , , , , | Permalink | Comments (6)
Business Cocoa Development Objective C Personal Tech

Introducing: SourceTree

October 26, 2010 – 4:24 pm

I’m pleased to announce that I’m finally ready to make my first fully-fledged commercial Mac OS X application available to the world!

SourceTree is a user-friendly Mac OS X front-end for Mercurial and Git, the two most popular distributed version control systems used today. The goal was to create a single tool which could deal with both systems efficiently, and to give a developer quick and intuitive access to the things (s)he needs to just get on with building software.

I thought I’d answer a few background questions on this that I get asked on occasion:

Why Mercurial AND Git?

Other apps tend to concentrate on just one version control system, so why am I supporting two? Well, as a developer I’m regularly coming across projects from both sides of the fence, and in practice I find I need to use both fairly regularly. I personally chose Mercurial for my own projects (and discussed why here), but I still use Git when dealing with other projects, and spend a fair amount of time hopping between the two. It struck me that even though they have their differences, they are both based on the same distributed principles, so having to use two separate tools was just unnecessary. I wanted a single tool which provided a common interface where that made sense, while still exposing the things they do differently where that was useful too. SourceTree 1.0 is my first attempt at that.

Why only Mac OS X?

There were actually multiple reasons for this choice:

  1. I wanted to learn Objective-C and Cocoa on a real project
  2. I know from experience that designing for multiple platforms can be a distraction, with more time spent on compatibility issues, and less on functionality – and that’s before you even consider the compromises  you have to make, particularly on UI conventions which are far from uniform across platforms. I’ve been a multi-platform developer for more than 10 years, and for a change I just wanted to focus on the end user results and nothing else. I’m aware that schedules slip very easily when you overcomplicate, and I’m already supporting multiple DVCS systems (something I consider to be an important feature point), so I deliberately chose to keep this element simple.
  3. Mac OS X has become my own platform of choice for most things now. The combination of stability, user-friendliness, Unix underpinnings and well designed hardware match my current needs perfectly. I’m done with the ‘some assembly required’ PCs that I loved tinkering with over the past 15 years

What about Subversion?

A few people have asked me if I plan to add Subversion support too. I actually did intend to originally, until I realised how much time it was going to take to just do a decent job on Mercurial and Git. Within the time constraints, I focussed on the subject areas that I felt I could contribute most to – there are already quite a few Subversion tools out there for Mac OS X, but Mercurial and Git are much less well served, so that’s where I focussed my efforts.

I still have Subversion support tentatively on my work plan, but it’s not top of the list. I think it’s better to do your most important features well before diversifying. Plus, there are problems with Subversion – it’s very, very slow compared to Mercurial and Git, so to match the performance in SourceTree of things like the interactive searches and dynamic refreshing / log population I’d probably have to do a ton of extra caching just so the user wasn’t sat tapping their fingers.

Edit: I made my decision on this: I don’t plan to support local Subversion, but to support operating with Subversion servers with Mercurial and Git locally via hgsvn and git-svn.

Why didn’t you make it open source?

Sorry folks, while I love contributing to open source (I’ve done a bit on SourceTree too, sending a patch back to BWToolkit), making it work as a business is very hard indeed. I half-killed myself trying to combine being an open source project leader and doing other commercial activities at the same time, so now I’m trying a more traditional approach. One thing I learned in the last few years is that there are some sectors & application types where being an open source maintainer is very compatible with also running a business based on that project, and there are others where you can really only do one or the other simultaneously without flaming out. Sucks, but there it is ;)

What’s Next?

I have a public, official roadmap for SourceTree and encourage users to suggest things they think should be on there, via the support system. I learned from running an open source project for 10 years that being open about your plans can be a big benefit – users like to know where things are likely to be going, and often have better ideas than the developer on what could do with a bit more spit and polish. They can also tell you what’s important to them, which is crucial for prioritising – as developers we tend to get carried away with things we want to work on, but in the end, it’s scratching the customer’s itch that matters most.

And while I’m really quite proud of SourceTree 1.0, there are plenty of features I’d like to continue to add, and definitely more room for some totally unnecessary beautification which I didn’t have time for in the first release. Hey, this is OS X ;)

SourceTree is available now on a 21-day trial license. Go get it already :)

Tags , , , , , , , , | Permalink | Comments (14)
Internet Tech Web

Hosting services: my recommendations

July 8, 2010 – 9:32 am

After hearing on Twitter how an acquaintance’s new hosting provider went ‘mammaries skyward’ this week, much to their understandable annoyance, it occurred to me that I have some recommendations I can make on this subject. While I don’t host that many sites, I’ve been doing it for long enough and had experience of both personal and medium-traffic sites that I’ve experienced the highs and lows quite a few times already.

The Golden Rule: Support > Everything

When it comes to hosting, the most important thing to look for, beyond what all the statistics of how much space and bandwidth you get, beyond even quoted up-times, is the quality of the support service. The big question is: when things go wrong – and if you host long enough, eventually they will even in the best possible hosting environment – how quickly are problems resolved, and how responsive are the support engineers during the process. Literally nothing is more important than this, and unfortunately it’s the one thing that you’ll only really learn with experience, unless you’re hosting a site big enough that you can get a formal SLA. Assuming you’re not going big enough for that, the only way to judge this is by being with a provider for a while, or knowing someone who has been with them, or possibly looking at online review sites – although frankly these are often highly unreliable, polluted as they are with inaccuracies and omissions either because of ignorance (people who post glowing reviews after being with the site 2 weeks) and unfortunately by frequent shill reviews.

I’ll post a couple of hosts I’ve had good experience with over many years later in the post, through good times and bad.

Know Your Bandwidth

Personally, I instantly rule out any host that claims ‘unlimited bandwith’. This is a crock – there is no such thing, and to claim there is just means the host is already lying to you before you even start – they have to pay for their bandwidth, so they can’t possibly allow everyone truly unlimited bandwidth and stay in business. If you really need unlimited bandwidth, i.e. you have a high-traffic site with lots of media files, then you will quickly bump into the way that these sites offer this ‘unlimited’ deal – via throttling. You may not have an absolute physical cap on your bandwidth, but if the tap is locked off to a slow dribble beyond a certain usage, it’s really worthless. In practice, ‘unlimited bandwidth’ is just a marketing point that they hope will draw in people who will only actually use a tiny amount of bandwidth, but will somehow favour them because the offer looks good. Don’t be one of those dumb guys.

Really you need to establish your bandwidth requirements and head for a host that can fulfil them for a reasonable price. For example, ogre3d.org uses between 125GB and 250GB per month, which is a reasonable amount, compared to my personal site here which only needs 5-10GB per month.

If you have ‘spiky’ bandwidth, i.e. occasionally you need to be able to distribute large amounts of data, but it’s not a constant stream, it would be best to go for a lower monthly limit and host high-bandwidth items elsewhere. I often use Amazon S3 for this purpose which can be made to look like a sub-domain of your own site, and which charges for bandwidth at a very fine granularity so matches your demand closely – it’s more expensive than buying a monthly allowance if you use it a lot, but for on-demand spikes it works very well.

Shared, Dedicated or Virtual/Cloud?

I currently use two shared hosts and one dedicated host, to match the demands of each site. Personally, I’m still very skeptical about virtual private servers and cloud hosting, due to a bad experience I had a few years back when we tried running ogre3d.org on a VPS. We lasted not much more than a month before we moved the server to a dedicated machine because the VPS simply didn’t deliver on its promises – performance was unpredictable and to be honest you had the worst of both worlds – you had to admin your own server but you still didn’t have a 100% guarantee that no-one else would be screwing with something on the machine, or that the disk arrays wouldn’t be hammered by someone else (regardless of CPU assignment), or some other balancing issue. Virtualisation has evolved in the last few years so this may not be an accurate representation anymore, but personally I wouldn’t go for a VPS again any time soon, unless it was a machine I controlled entirely and partitioned myself into virtuals – at least with shared and dedicated servers you know exactly what you’re getting – either a low-maintenance but shared resource environment, or total control & power. VPS claims to offer a middle ground but in my experience it didn’t deliver.

So, who do I use?

For my shared hosting, I’ve been using Hosting Matters for about 10 years now. I went through a couple of other hosts before them and had terrible experiences, but since I switched to them I’ve been very happy. I can count the number of hours downtime (that I’ve been aware of) over those years on one hand, and whenever there’s an issue they’re incredibly fast to respond – they have both community forums and support tickets depending on the urgency. It’s also very reassuring to see the same names cropping up in the support responses over the years.

Their offerings are pretty standard, nothing that would make them jump off the page for anyone looking for a stellar feature list or super-cheap pricing. But they’re very reasonable, they’re honest about what they’re offering (like bandwidth), and as I said before, support > everything.

For dedicated hosting, since 2007 I’ve used Dedipower. They’re based in Reading, their support staff are all local and are on the end of a phone if you need them (no call centres). Having been through a UK dedicated server comparison twice in the last 3 years (once again just recently), Dedipower came out as the most competitive for the service they were willing to offer, and I’ve been happy with the support service. In once instance in fact, when I moved a sub-site off the server, they were quickly on the phone to me within 10 minutes to tell me it was ‘down’ – at which point I had to explain it was expected & apologise for not notifying them in advance. You really can’t complain about that.

I hope that’s useful to someone. In case I need to point this out, I’m not getting paid or receiving discounts to promote either of these hosts, they’re just the two I’ve been most happy with over the ~10 years I’ve been hosting sites. YMMV but they’ve worked well for me :)

Tags , , , , | Permalink | Comments (14)

Next Page »

Next Page »