Because third-party applications – like Twitter and Coveritlive – are an important part of the service that we offer, it felt like it was time to try to explain our thinking on how we work with them, both how that works but also why it’s not always the easiest thing to do.
I’ve just got my copy of “Codermetrics: Analytics for Improving Software Teams“, by Jonathan Alexander. I’ve had a very quick skim through and initial thoughts are that this is going to be an interesting read and a worthwhile purchase.
I’ve been struggling with the best way to measure the efficiency and effectiveness of software development teams, having tried a number of approaches and found them lacking. The approach presented in “Codermetrics” is a novel approach and one which I am keen to evaluate.
The Codermetrics principal is not based on performance reviews (though the metrics can be used in performance reviews) – it tackles the issues at a lower level and in a way analogous to how the performance of members of sports teams is measured. As well as concentrating on individual performance, it includes metrics for an individual’s contribution to the team, their flashes of genius and their responsiveness to issues.
The metrics are grouped into 3 areas:
- Skill metrics (measuring core skills);
- Response metrics (analysing the ‘fall-out’ from a product release);
- Value metrics (combinations of the above to measure the values that individuals bring to the team).
The choice of skill metrics is, well, unique, in my experience, and this is where the analogy with sports teams is most obvious. I like the analogy, the metrics and the reasons given for choosing them.
Once I’ve had a good read through, I’ll provide a more thorough review. Suffice it to say that I’m really quite keen to try this out.
I’m looking to find a software company in the Sussex area (or London) that is using Agile development methodologies, and which wouldn’t mind accommodating me for perhaps a day to look at their processes in action. I’m particularly interested in:
- Scrum-based development
- A company that is not Rails/Grails based
- Is managing, to some extent one or more of the following:
- Automated testing,
- Automated deployment,
- Database migrations,
- Development management software (such as Pivotal Tracker, but not necessarily)
- Brighton area, but anywhere in Sussex or London would be great
I’m hoping that there are altruistic, sharing companies out there that will take me in and show off their practices. I’m keen to move our development further up the ladder to full ‘agility’ and would appreciate the time and others’ experiences in making the right choices.
Many thanks in advance.
Please contact me directly through any of the contact points on my about page.
Also, if this is not for you, but you know of someone who may be able to help, please send them a link to this post.
A long time ago I said that I would be talking about Key Performance Indicators (KPIs) for Software Development departments. What with various other commitments, holidays and the rest, it’s not happened, and looks likely to be happening on my company’s blog when it does.
That said, I do want to put down some thoughts regarding my experiences so far in trying to consolidate data from ‘the top’ (roadmaps & strategies) and ‘the bottom’ (timesheets) and getting them to meet in the middle (KPIs).
The essence of what I want to do with KPIs is measure accuracy, efficiency and cost. In order for any measurement to be done we need to capture ‘actuals’ – time accrued against work packages – and compare that to what we (I) estimated they would take. This necessarily leads to the need for all relevant parties to complete timesheets. Fortunately, this has been accepted in good spirit and, although I do have to provide a gentle reminder once in a while, I’m receiving completed timesheets. (Timesheets are paper-based. This was by choice, as we couldn’t find a suitable application that would meet our requirements and also because we wanted to trial and streamline the process/data before committing to any particular system.)
On the other side – the estimates – I have a rather natty spreadsheet on which I put requirements, break them down into work units / stories (with estimated effort) and assign them to staff. I can then assign each requirement to a Year and Quarter and watch pretty graphs go up and down with estimated capacity / slack in the department, which allows me to rehearse ‘what if’ scenarios in scheduling meetings. (Our strategies and roadmaps are ‘agile’. Are yours?)
My problem is getting the twain to meet. My experience thus far is:
- I can’t read other people’s writing.
- People aren’t necessarily following the instructions about how to fill out the sheets.
- Mistakes are made.
(The last one of the above is not specific to paper-based timesheets, but mistakes can be made in writing the sheet and in me reading the sheet. With an application-based solution there would only be one human in the process.)
So far, getting the timesheet data into my all-singing spreadsheet has proved both time-consuming and though-provoking; the main thought being “there must be a better way”.
In a nutshell, I really need to get it all automated. This is not going to be easy as I don’t think there’s anything in the market that will do everything I want, and even a mash-up will need a central co-ordination hub. I would love to be able to piece already existing applications together and will be investigating solutions for each tier of the process, but I do believe that there will be large chunks that won’t be available or, if they are, won’t be suitable.
So, I’m going to start a ‘hobby’ project, concentrating on the aspects that are most pressing – getting timesheet information into a database that I can use to compare against what I already know from my spreadsheet. This will involve investigating the timesheet facilities available in other intranet applications we use (our CRM has timesheet functionality, for instance).
If I can borrow or build the timesheet functionality I’ll expand upon that to incorporate stories, requirements, products etc. into a coherent, strategic software development management application, again integrating external systems (Pivotal Tracker / Lighthouse / our CRM) where possible.
My head’s buzzing with ideas, but I need to stay focussed on the essential. I’ll keep you up-to-date with developments.
There is a lot of resistance to timesheeting, and this appears particularly to be the case amongst development teams. As an ex-developer (and still occasional developer), myself, I remember being in the same camp. But as I’ve progressed (upwards, for the most part) and have to start accounting for my department’s cost, I have concluded that the only sensible way to gain useful metrics is via employee timesheets. Timesheets, alone, however, don’t provide all the information that I (or my superiors) require, and so should be used in conjunction with other systems and tools.
Having moved from being dead-set against timesheets in my early career, towards understanding their benefits and, ultimately, near-evangelising about them, I believe that the single most common reason why they are not so readily adopted is a matter of communication.
So what are the detractors saying? Here are some of the most common issues that I have encountered:
- It’s big brother – I don’t want my every move recorded.
- At the end of the day I can’t remember / don’t have time to record everything that I did.
- It will show that I don’t do a full day’s work & I’ll be penalised / pay docked / flogged.
- It will highlight the days when I am late for work.
I can understand all of these – I’ve felt them myself.
So what’s good about it? Here goes, in no particular order:
- No-one (at least no-one who understands how people work) expects you to put in a full day’s productive work. It’s not possible. You make tea, have a cigarette, read emails, attend meetings, get distracted by a conversation that someone else is having. These are all times when you are not ‘being productive’. But, it would be an awful place to work where you couldn’t do any of the above and were expected to be productive from the second you sat in your chair until the second you could go home. That would be a Victorian work house; you don’t work there.
In the more progressive working environments, bean bags, pool tables and games consoles are available; it’s understood that people are more effective when they are able to step away for a break – they get more done during their ‘productive’ times.
If I see a timesheet and on it Bill says that he’s spent 8 hours developing ‘Product X’, then there’s a problem; it simply can’t be true.
- So what if you get into work late? We don’t expect you to be 100% productive (see above), so the odd half hour here or there isn’t going to break the bank. If it’s going to be a regular occurrence, make an arrangement with your line manager. (The issue is not that you are late – it’s that other people see you coming in late and ‘getting away with it’. If you have made an arrangement with your manager then they can rebut any ‘tuts’ etc.) The one thing you absolutely must do, though, is apologise for being late (in the case where it’s not pre-arranged), but that’s just politeness.
As an aside, I think it’s worth stating at this point that there are two ways of implementing a timesheet system. One that tells you what a particular person was doing at a particular time of the day; and a second that simply tells you how much time was spent on each task during the day. It’s the latter that I would always advocate; It resolves a lot of the issue with people remembering what they did when the end of the day arrives and it’s quicker to enter information in this fashion. It’s true that the information entered is likely to be less accurate than ‘at the time’ recording of the information, but these things average themselves out in the long run. It also depends a lot on the granularity of your system – whether you record by the hour, half hour or quarter hour, for instance.
To a certain extent, timesheets are big-brotherish. That’s kind of the point of them. However, I see this as a good thing, rather than a bad thing. To the person who says to me “It’ll show that I didn’t work full-time on ‘Project X'”, I’ll say, “Yes, and it’ll show why.” – When my boss says “You said you’d deliver Project X by today – why is it going to be 2 days late?”, I can say “Because Bill spent 10 hours working on that little side-project you insisted on chucking into the mix.” (Although, as a good manager, I would have flagged this far sooner than d-day, of course.)
I need to know what my team are working on. I can’t sit over their shoulders 24/7 cracking a whip, I certainly don’t want them to have to work like that, and I can’t be a constant gate-keeper, making sure that nothing off-piste gets added to their work lists. But at the end of the day (both literally and metaphorically) I want to know that these things have happened and how much of a distraction they were. With this information I can better estimate timescales, costs, schedules, etc. And that’s a good thing. It’s the very nature of needing to know all these asides and distractions that makes the implementation of a timesheet system so compelling.
Rather than a conclusion, I thought I’d share my thoughts on where this is going. Firstly, I’ll be looking into ETT – Emergent Task Timing and will feed back on whether that might provide the compromise between accuracy and efficiency with regards to timesheets.
Next up will be about metrics, statistics, and KPIs. Timesheets and other tools will be involved.
You have been warned.