Saturday 2 January 2010

What did I learn in 2009?

I attended both Software Craftsmanship conferences and as a result I now see software craftsmanship as a 'movement' in the same sense as Impressionism. In other words it's a group of very loosely affiliated people who are linked by overlapping values rather than rigid adherence to some set of rules. However its roots in the Agile movement mean its likely to be subject to the same forces that took us from 'lightweight methodologies' for getting useful work done to a marketing brand with a formal alliance overseeing its usage. Large chunks of the Agile community seem to have become more interested in process improvement than in improving their skill or their techniques. Reading parody sites like this: and finally asking awkward questions about Scrum (it's a very bad sign when no-one will publicly answer a question like "is it possible to fail a Scrum course?") made me realise that 'Agile' has become a meaningless tag that I don't want to be associated with any more.

This was the year I started to really see the limits of my tools and techniques. This includes Java, TDD, the LAMP stack and the conventional approach to logging. As a result I'm doing more work with Python; thinking more in terms of invariants and algorithmic complexity as well as looking at tools like QuickCheck and SparseCheck; playing with App Engine and experimenting with the event sink approach that Nat Pryce and Steve Freeman describe in their book Growing Object Oriented Software.

The aforementioned limits (plus a discussion with Dan Creswell and Manik Surtani) lead me to the Scalability Staircase and I realised that much of my 'experience' was now invalid because I'm not working in an enterprise environment. In fact many of those enterprise software environments have radically changed but the people working in them are only just starting to realise that they'll need different approaches to software development now that they're working at web scale.

I started to appreciate the benefits of feature segmentation/partitioning through flags and dependency injection. Flickr wrote a somewhat controversial article about their approach although many of the critics haven't spotted the connection with bucket testing/multivariate testing.

In 2009 I found that I got better results by modelling people who were effective rather than brilliant. Effective people tended to have generated a lot of artefacts whilst the brilliant tended to agonise over a small set of projects and as such artificially constrained their potential impact. Perfectionism often got in the way of delivering results whilst focussing on getting a little better every day left with me lots of useful artefacts, lots of real-world feedback and more opportunities to take creative risks.

Getting in the habit of making improvements rather than offering criticism leads to lots of work but it also leads to things getting better. As a consequence I'm now spending more time helping out on open source projects. Another consequence was discovering that you're better off focussing on people's artefacts (their code, their data, the things they've made) rather than their opinions. Without that focus it becomes difficult to distinguish between people who have discovered better skills/techniques and those who just have a convincing argument.

One of the side-effects of my new-found appreciation for these 'productive communities' was that I attended more BarCamp-style events. At one of those I ran a session which gathered a group of people who understand the semantic web and Linked Data. Thanks to those people I now understand the appeal of Web Scale Identifiers. Basically the idea is that if your website is about something like books, music, musicians, etc which already has a good identifier (e.g. ISBN, ASIN or MusicBrainz id) then you should re-use that identifier. Inventing your own identifier or exposing your database's primary key or using the simple name leads to a wide range of annoying problems for your users whilst re-using existing identifiers means you get to benefit from the work being by the rest of the Web.