Work 2.0 – the interruptible programmer

I’m 37, and I’ve been a (professional) developer for 16 years. You would have thought that in that time, I’d have figured out an effective work style which delivered the desired outcomes (code cut, products shipped etc) without causing detrimental knock-on effects – but, sadly, you’d be wrong. I think the style in which I practiced my craft for the first 15 years of my career was much the same as every other enthusiastic developer: you put a ton of hours in. 12-16+ hour days, evening and weekend coding marathons, pizza in the keyboard, crunch times, 3am debugging sessions where you just can’t go to bed because you can feel the source of that bug just beyond your fingertips, dammit, desperate last-minute sprints to deadlines where you manage to slot that last piece in, Jack Bauer-like, just before the world goes to hell. If you’re in the demographic I’m talking about, you’re nodding sagely, and probably grinning a little too, reminiscing on past trials and glories. This sort of crazy dedication is respected in our circles, and is pretty much expected of any developer who has claimed to earn their stripes.

But, it turns out this kind of thing is not good for your health – who knew? Those of you who know me or keep up with my blog know that I’ve been dragged kicking and screaming away from my old ways, because of back issues that I initially ignored, then tried to cope with using token accommodations, and finally succumbed to in a big way. Being self-employed, this was a major problem. Crawling out of the pit I dug for myself took a long time and a lot of frustration – I read quite a few productivity books on the subject to try to find answers on how to keep working, and in the end found that the answers you mould for yourself tend to be the best ones. I’d like to share one of the things I learned along the way.

But I’m ‘In The Zone’!!

So, I want to talk about the biggest problem I encountered: concentration periods. I can’t sit at a desk for longer than about an hour at a time now; if I don’t get up and walk around, do some gentle stretching etc, at least this often, I’ll pay for it badly once I do move, and probably over the next few days too. I also can’t realistically work more than a standard 8 hour day without pain any more. The problem with this was that, as a programmer, the style which I developed over 15+ years involved getting gradually ‘Into The Zone’ and coding for very long periods at a time, uninterrupted. This is a common theme among coders, who like to shut themselves away for hours at a time, wear headphones to avoid distractions, have ‘quiet times’ and so on – and it’s also why we tend to react really badly when interrupted. Programming requires concentration, and concentration seems to run on a valve system – it takes time to warm up, and once it’s going, you don’t want to turn it off because starting it up again is a major hassle.

I thought there was no way around this, and had begun to resign myself to just being less productive because of it. However, over the last 6  months in particular, I’ve discovered that, far from being an intractable problem, this ‘slow warm up, long uninterrupted focus time’ approach is to a large degree a learned behaviour, and it’s possible to re-train yourself to cope with things differently. It’s a little like when people learn to adopt polyphasic sleep patterns – it’s not that you can’t do it, it’s just that when you’ve become accustomed to doing things a certain way, changing that is initially very, very hard. But it’s not impossible, given the right amount of motivation and time to adjust.

So, my goal was to acclimatise myself to many shorter work chunks during the day instead of a few very large ones, while still maintaining productivity. The key to this was to learn how to get back ‘In The Zone’ in the shortest time possible – much like the way polyphasic sleepers train themselves to achieve REM sleep more quickly. I’m mostly there now, or at least way better at it than I was, so, what techniques did I use to make this transition?

  1. Embrace interruptions
    This is less of a technique and more of a deliberate psychological adjustment which cuts across all the practical approaches I’ll cover next. Instead of being the typical coder who avoids interruptions at all costs, you need to accept them, and learn to manage them better. It’s hard – you have to try to set aside years of resisting interruptions and initially, until you adjust, you’ll feel like you can’t get enough done. Many people will probably want to give up, unless there’s something specific motivating them to push through it – for me, daily pain was a great motivator. My main message here is that the transition is just a phase, and that it is possible to be an interruptable programmer who still gets things done. But you have to learn not to fight against it, hence why this is the first point.
  2. Maintain context outside of your head at all times
    Much of the problem with interruptions is that of losing context. When you’re in that Zone, you’re juggling a whole bunch of context in your head, adjusting it on the fly, and maintaining and tweaking connections between issues constantly. Interruptions make you drop all that, and it takes time to pick it all up again. My answer to this was to externalise as much as possible, on as many levels as possible:

    1. Maintain a running commentary on your current task
      I am my very own chronicler.
      I write notes on what I’m doing all the time, whether it’s adding a comment line to a ticket, committing frequently and writing detailed commit notes (you do use a DVCS to make light commits more practical, right? ;)) scribbling a drawing on (ordered) pieces of paper. This really isn’t that onerous, and in fact externalising your thoughts can often help you clarify them. Basically the guide is that roughly every 30 minutes, I should have generated some new piece of context which is stored somewhere other than my head. If I haven’t, then that’s context I’d have more trouble re-building mentally if I’m interrupted. It doesn’t take much time to do, and it has other benefits too such as recording your thought & decision process.
    2. Ruthlessly ignore tangental issues
      You might have noticed that in the last bullet, I used the words ‘current task’, singular. Not ‘tasks’. There is no such thing as having more than one ‘current task’ – there is only the one task you’re actually working on, and distractions.
      We probably all use bug trackers / ticket systems to track bugs and feature requests, but when you’re working on a ticket, it’s very common to spot a new bug, or identify an opportunity for improvement, or think of a cool new feature. How many of us go ahead and deal with that right away, because it’s in the area we’re already in, or it’s ‘trivial’, or it’s a cool idea that you want to try right now?  I know I did – but I don’t any more; any tangental issues not related to what I’m currently doing get dumped into the ticket system and immediately forgotten until I’m done with the current task, regardless of their size, relevance or priority. It sounds simple and obvious, and this might even be official procedure in your organisation, but I challenge most coders to say that they actually do this all the time. The benefit is that even the tiniest of distractions add an extra level of context that you have to maintain, which is then harder to pick up again after an interruption.
      For this to work, you need a ticket system which is fast, lightweight, and doesn’t require you to be anal about how much detail you put in initially. You need to be in & out of there in 30 seconds so you can offload that thought without getting distracted – you can flesh it out later.
    3. Always know what you’re doing next
      This is one from GTD (‘Next actions’), but it’s a good one. When you come back from a break or interruption, you should spend no time at all figuring out what you need to be doing next. Your ticket system will help you here, and so will the running commentary that hopefully you’ve been keeping on your active task. If you’ve been forced to switch gears or projects, so long as you’ve maintained this external context universally, you should have no issue knowing what the next actions on each item are. The important thing is to have one next action on each project. If you have several, you’ll have to spend time choosing between them, and that’s wasted time (see the next section on prioritisation). At any one time, you should not only have just one current task, but one unambiguous next action on that task. Half the problem of working effectively is knowing what you’re doing next.
  3. Prioritise Negatively
    I mentioned next actions in the previous section, but how do you decide what comes next? A lot of time can be frittered away agonising over priorities, and I used to struggle with it; I would plan on the assumption that I wanted to do everything on the list, and I just needed to figure out which I needed to do first. I discovered that I could cut the amount of time I spent on planning, and also get better, less ambiguous priorities by inverting the decision making process – to assume a baseline that I wouldn’t do any of the tasks, and assessing the negative outcomes of not doing each one. So instead of ‘which of feature A or B is more important to have?’, it became ‘Let’s assume we ship without feature A and B. What are the issues caused by omitting them in each case?’. It might appear to be a subtle difference, but having to justify inclusion entirely, rather than trying to establish a relative ordering assuming they all get done eventually, tends to tease out more frank evaluations in my experience.
  4. Recognise the benefits of breaks
    Much of the above is about limiting the negative aspects of taking breaks, but the fact is, that they have many work-related benefits too. I’m willing to bet that all coders have stayed late at work, or late into the night, trying to fix a problem, only to find that they fix it within 15 minutes the next day, or think of the answer in some unlikely place like the shower. The reason for this is very simple – extended periods of concentration seem productive, and can be on operational / sequential thinking, but for anything else such as creative thinking or problem solving, it’s very often exactly the opposite. Not only do tired minds think less clearly, but often the answer to a problem lies not in more extensive thinking down the current path which you’ve been exploring in vain for the last few hours, but in looking at the problem from a completely different perspective. Long periods of concentration tend to ‘lock in’ current trains of thought, making inspiration and strokes of genius all too rare. Creativity always happens when you’re not trying, and it’s an often under-appreciated but vital element of the programming toolbox. Interrupting that train of thought can actually be a very good thing indeed.

There’s more I could talk about, but that’s quite enough for now I think. I hope someone finds this interesting or useful :)

  • terry

    I ABSOLUTELY LOVE this line:
    “There is no such thing as having more than one ?current task? ? there is only the one task you?re actually working on, and distractions.”
    I’m just wondering when the truth is finally gonna get out that multitasking is ONE BIG STEAMING pile of BS created by business types who are trying to “maximize efficiency” or whatever new buzzword they came up with this week. Good post. Everybody who reads this spread the word about the fallacy of multitasking. The truth shall set you free.

  • san_kan1gb

    Thanx, it was very beneficial.

  • Pingback: links for 2010-10-16 « pabloidz

  • mr b

    You can tell how immature our industry is by the glamorisation of working long, unsociable hours. We are the fools for doing it because you only get one shot at life and a successful person is a one that protects there health and finds balance in there live. What other occupation ask you to sacrifice your health and happiness? There also needs to be a recognition, mainly by management types, that ours is a job whereby we need to minimise noise and disruptions. Having said that being successful isn’t just about coding lines of code. Interruptions are an important part of the day to build up relationships with fellow colleagues, customers etc.

  • Anand

    Thank you for the great post! I use text files to keep track of the ‘running commentaries’. :)

  • Cameron Young

    I was talking with a friend who is a civil engineer for major dams and flood control systems. We were comparing work hours. His routine was a consistent 40 hours/week maximum, with the 40 hours spent productively. He spends another 8 hours per week of his own time reading technical papers to maintain his skills.

    When he heard how many hours I worked (60 sometimes up to 80) his comment was “Well then you musn’t do anything very important.”

    His comment nagged at me because I realized he was right. His work must be free of any significant mistakes over his entire career. I live with many small mistakes due to the continuous pressure to meet deadlines.

  • http://www.nikoslianeris.gr nikos lianeris

    very nice article.And really useful and inspiring for a noobe developer-like me- :)

  • Guy

    Your experience resonates with me too. I agree with each of your points, but find embracing interruptions from people too lazy to think for themselves, infuriating. I recently converted to the Pomodoro technique. It has noticeably improved my productivity as well as reducing neck and shoulder strain. As other comments allude, regular exercise had a profound effect on my life. Yes it’s hard to start with, but achievable goals and making it fun were the keys for me. I find good habits easier if incorporated into my routine. I’m just a creature of habit.

  • ts

    Hi,

    thanx, nice other ageing :-) programmers have similar problems. I can subscribe to most of that, except that i
    `m to chaotic to keep nice records. but
    1: take some exercise. Gym sounds boring, but 4h a week are well spent, considering the pain and costs of all those back issues you mention. And think: All of us need to keep on hacking until 6x (67 in GE, actually)

    2: (contradictionary..) Good ideas come the next day, or under the shower, agreed. But not without banging your brain before for a while. (Edison: Genious is 95% sweat or so…) But age enable you to figure out when it was enought, not like all the youngsters. Thats our advantage, and we must take it :-)

  • ar

    excellent article :)

  • Pingback: Michiel . van Vlaardingen » Blog Archive » Getting Things Done: The Pomodoro Way

  • http://www.synthesis.my alien3d

    I’m jogging on each weakend if free. sometimes the body need time to break.

  • M Towler

    Excellent article, the approaches mentioned have many similarities with David Allen’s excellent book “Getting things done”. I cannot recommend it highly enough.

    http://www.amazon.com/Getting-Things-Done-Stress-Free-Productivity/dp/0142000280/ref=sr_1_1?ie=UTF8&qid=1287399734&sr=8-1

  • http://www.stevestreeting.com Steve

    @M Towler: Thanks, and the similarities are no co-incidence, GTD was one of the books I read in the last couple of years while searching for solutions. :)

  • http://ohai.im/joe.klemmer Joe Klemmer

    Stumbling across this post was quite fortuitous for me. I, too, suffer from physical issues that cause pain. What I have been doing is just pushing through it. But, when I look at my productivity objectively, I know I’m not getting done anything near what I could be doing.

    This work pattern doesn’t completely fit my overall situation (my biggest interruptions come from all the pain meds and anti-depressants I’m on) but it does give a great framework to build from. And having that foundation is really the most important part.

    Thank you for this.

  • Vitor Castelo

    Hi all! Excelent article Steve.

  • http://szeryf.wordpress.com/ szeryf

    Reducing the cost of interruptions is also what Test Driven Development is about on a deeper level. When you only write test or production code in minimal increases, you don’t leave too many open loops when interrupted.

  • http://www.parasoft.com Erika

    Great comments! We’ve done a ton of R & D on TDD, here is a whitepaper if you’re interested: http://www.parasoft.com/testdrivendevelopment

  • syd

    Very interesting!
    and glad to read an optimistic post from you steve.

    I’ve always needed a 10 minutes break every 1.5 hours, to breath fresh air, walk around…
    Coders are single threaded, and need a good cooling system :)

  • Pingback: Weekend miscellany — The Endeavour

  • http://opengamestudio.org kornerr

    Thanks for the great article. In USSR there was Lubishev A. A. who took a bit deeper approach to himself – he has been logging where he spends his life time into for 56 years until his death. I’ve decided to take similar approach to find out what time I can use to progress in Open source. His method has one single big connection with your article – usage of many small chunks of time. For example, he learned English while taking bus rides.
    I’ve also translated your article into Russian: http://kornerr.blogspot.com/2010/10/20.html
    using this technique of small chunks of time. I’ve been spending 10 to 15 minute chunks for 18 days. And in the end I’ve spent 9h 35m to translate and revise it. The system really works and feels lightweight.
    Thank you.

  • kinjalkishor

    @ steve, polyphasic sleep patterns while may sound good and may feel cool for career growth, the fact is tampering with normal sleep is a killer as body recovers from injuries while sleep and repeat small injuries if not cured will grow exponentially and will break the person soon enough.

    I am myself sleeping at 1-2 pm regularly and waking at 7:30 am, thus getting less sleep(I normally sleep at 10:30-11:00 pm) for past 3 months. And the result is, while previously my regular exercise schedule was easy and strengthened my body, Now It seems difficult everyday and my main concern these days is to avoid injuries(like stretches and cramps) and to lower the intensity of exercise to cure the injuries I get. I have stretched my left shoulder twice, left upper arm once(long pain) and my left elbow once in past 2 months, on the exercise which was easy previously and not demanding. I feel a rise of acidity and general weakness and tiredness whole day. I am going back to normal sleep time as soon as possible once I finish some very important work.

    @ kornerr, usage of many small chunks of time, is nothing new, good students practice it all the time, (there was one student who studied even in the market and he managed to get in topmost technical institute of India also).
    Usage of many small chunks of time, requires great dedication and good mental discipline which many people seem to not able to do. For ex., Steve neglected his back for long and could not quite pull himself up to cut his work hours as he enjoyed them so much. I myself never did regular study in school.
    On the other hand I have learnt and thought so much in these “small chunks of time” and so much work is done in that, it is almost staggering. My FPS game is being written by me all in extra small chunk times and learning asociated with it(open source helps so much, or I would have not been able to figure out so many things).

  • kinjalkishor

    @ Joe Klemmer, at your age and condition you might want to put a little less effort and give more time to health as maintaining health is alpha prime and as you surely have noticed helth disbalance is affecting far more on your productivity.

  • kinjalkishor

    @ David, Thanks for pointing out Vitamin D info, very very helpful.

  • http://www.stevestreeting.com Steve

    @kornerr: excellent – I don’t think I’ve had one of my blog posts translated before (to my knowledge), and very interesting to hear the way you did it.

  • Ivan

    Many thanks for this post and especially for the idea of externalized context. That was the last peace I was in need to build my own system of interruption-friendly work. In fact it is quite similar to yours :)

  • Jonas

    Thank you for a great and inspiring post!
    This is something I really need to improve on, so far I’m down with the first point and last point, but when it comes to the second, “Maintain context outside of your head at all times” I’m finding some trouble.
    I’m struggling to maintain a running commentary on my current task, but I find too often that I don’t know what to write, or in what level of detail. Would it be possible for you to provide some sort of example, or sample out of your commentary?
    Any tips for tools that would help is also welcome, I’m currently using vim and creating files with the current date as name.

  • Pingback: The Distracted Programmer « Streaming Colour Studios

  • http://www.stevestreeting.com Steve

    @Jonas: sorry for missing your comment. I use 2 main things to capture my running commentary: Redmine tickets and pencil & paper. Whenever I have a thought on a project while working on something, I’ll push it into Redmine as a new ticket most of the time, even if that’s only one line. I’ll also write progress notes on things I’ve discovered and thoughts that I have on the current task.

    In contrast, if I’m just getting up to stretch and make a coffee, and it’s not a large amount of detail I need to capture, I’ll write down on a piece of scrap paper what the next thing I intended to do was (if I wasn’t taking a break), usually as a rudimentary mind-map (read: a couple of circles linked with lines mapping out what I was thinking about). When I come back, those scribbled few words / thought bubbles kick-start my thoughts back to where they were when I left, and Redmine will fill me in on the more extensive details if there are any significant ones.

  • Pingback: Sunetos, Inc. :: Think, Think, Think…

  • Phil

    I love the concept of externalizing the context. I have been moving in that direction to try to improve my productivity. I have to switch between projects and it takes me time to mentally switch. This in addition to interruptions. I will apply your recommendation.

    I used to work many hours, but I found that my judgement was suffering. Also I found that sleeping on the problem was more productive than hours of night work.

    Long hours are OK for short time, but not as a way of life.

  • Pingback: The Interruptible Programmer | Content Here

  • Pingback: thesecretdogproject » Running to keep up

  • http://orpheelafondlummis.net Orphée Lafond-Lummis

    Its great to see a respectable programmer like yourself doing such a feedback on the profession. As a young programmer, I find your ideas greatly inspiring and useful. Thank you!

  • Nate Reed

    I agree with your view on breaks. The human mind is capable of multidimensional thinking, but when we are caught up in trying to solve a problem we tend to be linear thinkers. When we stop working on a frustrating problem, it gives our minds the opportunity to process the data in new ways even while we are not consciously or deliberately trying.

    I have been thinking about this model of how our minds work and how we can be more effective and more productive programmers by tapping into this subconscious processing of information that is always going on in the background. We have massively parallel processors in our heads but how much of that potential is untapped?

  • Pingback: Indie and day job, making it work | David Amador

  • Pingback: Running to keep up | thesecretdogproject

  • Pingback: SteveStreeting.com » Oh no, not a ‘Best of’ collection?