Your Story Points are … ?

Welcome, This is a topic I’ve been wanting to explore for some time. It’s one that always comes up in many of the organisations that fly the Agile flag. None of these organisations seems to have cracked it yet either; enter… Estimation.

Now, I’ve seen a lot when it comes to Software Estimation. I’ve estimated in points, I’ve estimated in hours, days even t-shirt sizes. Recently, I was in a meeting where the team debated estimating in points AND hours. So, what’s a good techie to do?

You’ll all be familiar with this situation. A project is in it’s infancy and the Pointy Haired Bosses (PHB) want to know when the project is going to finish. So, where do you start? Ask the team, right? Well, when I was a Tech Lead reporting to a PHB, if a team member told me that thirteen points remained in the sprint, I’d think what does that mean to someone outside the project? As a team member I can hazard a guess that there’s a a sizeable chunk of work left? But to anyone not directly involved in the day to day work, it carries very little meaning.

In my experience, PHB is instantly going to add on thirteen days or two weeks to today, to come up with a delivery date. Then this whole awkward conversation begins about what a point estimate actually is, how a thirteen point story is actually more likely an “unknown quantity” depending on your scale; more and more people start glazing over, me included. PHB then raises concerns that delivery of said new feature is slipping, etc. and panic ensues.

You see, the thing with points is… they don’t mean time! Nor days or hours. They are not a scientific unit used to compare one team’s performance to another. They’re something else entirely — and different organisations draw different conclusions on what they represent. Different teams within organisations are often likely to calculate them differently too. It could be a measure of difficulty with a hint of time, complexity with a smattering of risk or just straight up a guess at a random number which roughly pertains to the size of the task and length of time it might take to deliver. Either way, this number should not be relied upon for anything.

I had a conversation recently and I genuinely wasn’t trying to be pedantic - it went a little like this:

PM: “is that a five or is it an eight?”

Me: I don’t really know

PM: Well, that one’s a three, is it the same as that or bigger?

Me: Well, they’re both really good numbers, but I’ve not actually started the task yet. So, I’m reluctant to say a number.

PM: So, shall we say a 5?

** PM referrers to anyone in a planning capacity

Why is this a bad conversation to have?

Timing

The timing of this conversation is crucial to its success. This conversation took place when the technical team where “blind” to what they had to implement. Let’s say it’s a Sprint Planning meeting and by blind I mean, this is likely the first time they’ll see some abstract Acceptance Criteria to base their estimate off. There was still a number of unknowns, architecture to be defined, dependencies to be resolved and spikes to be investigated, heck even the UX of the feature was missing in action. So, being asked to estimate or give a rough time frame at this stage of a project, is well, pointless… If you’ll pardon the pun!

Only once the problem is universally understood by all team members and sliced into manageable chunks can any semi-accurate estimating begin. Anything beforehand is complete and utter guesswork and carries no real meaning other than a finger in the air type estimate. Which begs the question, why bother?

Pressure

Estimates create pressure. They also represent a commitment, because most people will by default relate them to time even if they don’t mean to. Saying something is a 3 point story, this still creates a mental marker that this isn’t a big task and shouldn’t take the team long. But, what if for that 3 point story ends up taking 3 weeks due to a lot of firefighting, staff holidays and other unplanned work? That estimate was useless and cannot be used as a basis for future comparisons of tasks. If two 3 point stories are in a Sprint and one 3 pointer ends up taking about six hours to deliver because the team discovers it’s simpler than anticipated and the other takes 10 days — which was the right estimate? Now, you could argue that unplanned work and holidays should be factored into the plan to help prevent this kind of situation but the reality is, software development is hard and rarely goes to plan.

If the team completed 56 points in one sprint and the following sprint only completed 26, but successfully dealt with a major outage, does that mean the team didn’t deliver on their commitment?

In my experience, this kind of scenario is all too common and only serves to create a vicious circle of disdain from technical team members towards estimation and subsequent distrust from project planners towards the estimates given by technical staff. The whole system of estimating is flawed in that respect.

This process continues with Planners citing “We need to get better at estimating as a team” and Tech staff lambasting the practice of estimation. The team essentially creates a box ticking exercise for itself, a motion to go through to affirm the team is an “Agile Scrum Team”.

So who uses estimates anyway?

I’ve heard a number of people say:

“after a few sprints, we’ll be able to gauge the amount of work the team can commit to in a sprint, we’ll know our velocity!”.

Let’s be honest, to any PHB the amount of work the team commits to is: to finish what they started — to the point where there’s no more major bugs and the software is usable and yields a return on investment. Otherwise, why start in the first place?

The thing with all this estimating is… It really doesn’t matter. The team are going to carry on working on the project regardless, unless the project is canned and then, well, what help were your estimates in the first place? Features will keep on coming regardless of how they get into the Backlog either via PHB’s demands, Customer Research, Hackathons etc . There should always be a steady stream of things to work on. What matters is the timely delivery of these features and not overloading the delivery team.

I’ve come to the rather disrespectful conclusion that Story Points and Velocity et al are merely reporting mechanisms for middle management. When you think about it, like, really think about it. They don’t mean anything beyond the scope of the team, and dare I say, the people asking for them. Maybe, I’m working for the wrong people?

Having been both a Developer and a Senior Manager involved in project planning I’m yet to find a use for estimates that go beyond small, medium and large work items. Let’s explore this further.

• PHB doesn’t understand points and likely just wants to know a “delivery date”.
• The Product Manager or representative of the business will want to know when she gets her hands on the next iteration of the product to show off. Again, a date.
• Testers will want to push the software to breaking point and make sure there are no regressions. They’ll work towards making sure the product is fit for purpose in the time they’re permitted.
• The Tech Lead will be conscious of time and making sure the software is as robust and performant as possible.
• Business Users and other Stakeholders will likely just want to know - when?
• Sales / Marketing want to know when can they run their ad campaigns?
• C-Level Execs definitely just want to know - when and at what cost?

The Business Analyst is potentially the only stakeholder with a vested interest in the estimates. They know having large estimates leads to longer cycle times and won’t make their stakeholders happy, so it’s in their interest to keep the pieces small and the development pipeline moving.

So, what do we do instead then?

All right smart arse, So what do you propose instead?

I know I’ve bashed estimates a lot here. You may be agreeing with me or preparing your retort. Either way, I appreciate you sticking with me on this.

One of the most successful projects I’ve worked on involved a whole team of BA / Design / Devs and a Tester. The BA called a “pre-amigos” meeting and laid out plans for the feature to be developed. He had screenshots from the Designers, business rules he’d crafted along with data samples from the varying APIs used. Most importantly, he had a date which we needed to be live.

Collectively, the team walked through the design using the business rules as a guide and were able to see how everything fitted together. What happened next was great. We took that design and decided on three things:

• What was core to the new feature / what needed building first — the MVP?
• What was secondary, needs to be added, but not straight away?
• What bells and whistles could we add later?

With those three items decided, we then shifted our focus to try to determine some test cases. The Tester provided significant input here by saying what she would be looking for when the feature landed on her todo pile. The team used her input to determine the following:

• What scenarios could be tested using tooling & automation?
• What scenarios remained and needed a human?

This led to creation of a handful of automation stories which gave our Tester confidence that if these were all implemented, she didn’t have an awful lot left to do. We created approximately six automation tasks and one manual, full end-to-end testing task (this involved systems out of the team’s control for automation). Everyone was left feeling confident we’d covered all the scenarios. All sad path tests were able to be covered with automation too.

The meeting adjourned and the development team reconvened later that day to determine the architecture of the feature. Again, the team looked at the design and requirements from the Analyst to determine what order we should build things. We tried to make each task as small as possible. e.g. Add Feature Toggle, Add the Route, Add skeleton Controllers and Views, Add the page title, Add in Data from API call, Turn off Feature Toggle etc.

Each of the aforementioned tasks were individually shippable and took a few hours to implement. By having such small tasks the team felt like they were able to deliver value immediately. There was no talk of estimating: everyone on the team knew that each ticket had been broken down to its smallest releasable unit. The whole feature took less than a week to build and everything we shipped was code-reviewed (with a Developer and Tester) as well as showcased to Stakeholders. The team consisted of five people, one BA, one Designer/Frontend, two Developers and one Tester. We ate two pizzas for lunch every day that week.

The success of this project was down to the pre-amigos meeting where enough information was brought to the table to determine what needed to be delivered. Too often I see Technical teams left to drive out the details of feature requests made upon them and to provide estimates on criteria that were hastily determined moments prior. Rarely is a consideration for time spent on architecture included in an estimate. Occasionally, testing effort factored into the estimate — but only manual testing effort. Security very rarely makes the cut and Accessibility never gets a mention. Documentation… yeah right! Other *ilities too, are consigned to the bin when estimating. All of these things need to be taken into account when delivering new work; we can’t just slap a number on it and throw it over the fence to be implemented, box ticked. Teams that do this are missing out on some really great conversations and instead consigning themselves to providing estimates that don’t raise suspicion.

Pro-tip: Don’t estimate all your tickets as 1 point or 1 day — I saw a team do that once. Led to some awkward discussions!

Acceptance Criteria alone is not enough to bring to the table. Some people prefer visuals, however basic. Some prefer diagrams of architecture or information flows. Perhaps a mind-map, some stats, some customer feedback etc. Most importantly, have a date in mind when something needs to be out of the door. Armed with that, teams can begin to investigate what should be built and in what order and make their own calls as to what makes the MVP. Sometimes, it’s not possible to bring any of this to the table without the team first doing a spike into potential solutions or technology.

In Summary

Story Points aren’t important. In today’s day and age teams should be focused on acquiring knowledge of your customer’s needs, meeting those needs and validating your implementation. If you’re using Story Points as a way of sizing stories and not having collaborative conversations around architecture, testing etc. “you’re missing out”. I suggest you free yourself of the constraints of estimation and velocity and everything that comes with it by moving to an experimental approach: try something small, validate it, repeat. Getting things in front of your customers early is crucial to staying current.

In addition, team collaboration plays an important part of the delivery of software in large organisations. Use the techniques listed below and experiment with your team’s delivery and planning process, some may fit better then others, but all of them beat estimating.

Further Reading

• Lean Startup / The Startup Way — Eric Ries
• Continuous Delivery — Humble and Farley
• Lean Enterprise — Humble, Molesky, O’Reilly
• Accelerate — Humble and Forsgren

Techniques to Trial

• Example Mapping
• Event Storming
• Qualitative Cost of Delay Prioritisation