This post is (yet another) response to the McKinsey article “Yes, you can measure software developer productivity” Eagle-eyed readers might notice that while the title of his post is very similar to the original, it has one slight difference. I refer to “development” and not “developer” productivity. This is the key point for me. While there have been some excellent responses from Kent Beck and Gergely Orosz, and more recently Dan Terhorst-North, this is the point I want to emphasise in this post.
I have previously written about metrics and am a strong proponent of having data to guide improvement. Evidence is one of the elements of my TASTE model and the X-Matrix. So in all the justified backlash against measuring software developers, I don’t think we should throw the baby out with the bath water.
Measuring Outcomes
One of the mistakes that the McKinsey article makes is that it focusses on the developer. Thus it confuses productivity with being about developer activity. Dan describes this in much better detail than I can in his article. However, to summarise, productivity has nothing to do with how much code a developer can write.
Measuring the System
The follow-on from that is that productivity is more pertinent to teams, or teams of teams. Rather than measuring the activity of an individual, we should be measuring the outcome of the system. It doesn’t matter if every developer is cranking out hundreds and thousands of lines of code if the organisation is unable to get any of that code into production, and when it does it is full of bugs and delivers no value.
Balancing Metrics
That leads us to the fact that we shouldn’t overly focus on productivity alone. As I described in my earlier post on the evidence to look for in an agile transformation, I recommend 6 dimensions, of which productivity is just one. The other five are responsiveness, predictability, quality, sustainability and value. Focussing solely on productivity risks falling foul of Goodhart’s Law. This was popularised by Marilyn Strathern as “when a measure becomes a target, it ceases to be a good measure”. Having a balance of competing metrics can help avoid this. To be fair, the article does mention this at the end but it seems an afterthought.
Critical Factors
Having said all that, there was one thing that intrigued me.
I’m curious about the Developer Velocity Index benchmark. It has a horrendous name which I imagine puts a lot of people off even investigating it. However, it is described as something “which pinpoints the most critical factors (related to technology, working practices, and organizational enablement) in achieving Developer Velocity, as well as those that are not nearly as important as many executives and observers might believe”. With the caveat that we apply it to organisational productivity, I can imagine that being useful.
So, that’s my response. Just because we can measure software developer productivity, that does not mean we should. However, what is more important and more useful is to measure software development productivity, at the team, team of teams and organisational level.