“Codermetrics” – could be the solution I was looking for

I’ve just got my copy of “Codermetrics: Analytics for Improving Software Teams“, by Jonathan Alexander. I’ve had a very quick skim through and initial thoughts are that this is going to be an interesting read and a worthwhile purchase.

I’ve been struggling with the best way to measure the efficiency and effectiveness of software development teams, having tried a number of approaches and found them lacking. The approach presented in “Codermetrics” is a novel approach and one which I am keen to evaluate.

The Codermetrics principal is not based on performance reviews (though the metrics can be used in performance reviews) – it tackles the issues at a lower level and in a way analogous to how the performance of members of sports teams is measured. As well as concentrating on individual performance, it includes metrics for an individual’s contribution to the team, their flashes of genius and their responsiveness to issues.

The metrics are grouped into 3 areas:

  • Skill metrics (measuring core skills);
  • Response metrics (analysing the ‘fall-out’ from a product release);
  • Value metrics (combinations of the above to measure the values that individuals bring to the team).

The choice of skill metrics is, well, unique, in my experience, and this is where the analogy with sports teams is most obvious. I like the analogy, the metrics and the reasons given for choosing them.

Once I’ve had a good read through, I’ll provide a more thorough review. Suffice it to say that I’m really quite keen to try this out.


Gathering KPI data

A long time ago I said that I would be talking about Key Performance Indicators (KPIs) for Software Development departments. What with various other commitments, holidays and the rest, it’s not happened, and looks likely to be happening on my company’s blog when it does.

That said, I do want to put down some thoughts regarding my experiences so far in trying to consolidate data from ‘the top’ (roadmaps & strategies) and ‘the bottom’ (timesheets) and getting them to meet in the middle (KPIs).

Operator error

The essence of what I want to do with KPIs is measure accuracy, efficiency and cost. In order for any measurement to be done we need to capture ‘actuals’ – time accrued against work packages – and compare that to what we (I) estimated they would take. This necessarily leads to the need for all relevant parties to complete timesheets. Fortunately, this has been accepted in good spirit and, although I do have to provide a gentle reminder once in a while, I’m receiving completed timesheets. (Timesheets are paper-based. This was by choice, as we couldn’t find a suitable application that would meet our requirements and also because we wanted to trial and streamline the process/data before committing to any particular system.)

On the other side – the estimates – I have a rather natty spreadsheet on which I put requirements, break them down into work units / stories (with estimated effort) and assign them to staff. I can then assign each requirement to a Year and Quarter and watch pretty graphs go up and down with estimated capacity / slack in the department, which allows me to rehearse ‘what if’ scenarios in scheduling meetings. (Our strategies and roadmaps are ‘agile’. Are yours?)

My problem is getting the twain to meet. My experience thus far is:

  1. I can’t read other people’s writing.
  2. People aren’t necessarily following the instructions about how to fill out the sheets.
  3. Mistakes are made.

(The last one of the above is not specific to paper-based timesheets, but mistakes can be made in writing the sheet and in me reading the sheet. With an application-based solution there would only be one human in the process.)

So far, getting the timesheet data into my all-singing spreadsheet has proved both time-consuming and though-provoking; the main thought being “there must be a better way”.

Another way

In a nutshell, I really need to get it all automated. This is not going to be easy as I don’t think there’s anything in the market that will do everything I want, and even a mash-up will need a central co-ordination hub. I would love to be able to piece already existing applications together and will be investigating solutions for each tier of the process, but I do believe that there will be large chunks that won’t be available or, if they are, won’t be suitable.

First step

So, I’m going to start a ‘hobby’ project, concentrating on the aspects that are most pressing – getting timesheet information into a database that I can use to compare against what I already know from my spreadsheet. This will involve investigating the timesheet facilities available in other intranet applications we use (our CRM has timesheet functionality, for instance).

And then

If I can borrow or build the timesheet functionality I’ll expand upon that to incorporate stories, requirements, products etc. into a coherent, strategic software development management application, again integrating external systems (Pivotal Tracker / Lighthouse / our CRM) where possible.

My head’s buzzing with ideas, but I need to stay focussed on the essential. I’ll keep you up-to-date with developments.