True agility comes from listening to your customers and conducting experiments to find that product-market fit

Let’s begin by picturing the scene. It’s the development team’s retrospective, and team members are frantically scribbling their contributions on Post-It Notes themed into the familiar brackets of “Mad, Sad, and Glad” or “Keep, Improve, Stop, Start,” etc. We’ve all been there, right?

Once the arbitrary period of scribbling has finished, the retrospective facilitator has the task of grouping the notes into themes. The team then usually votes on the particular themes they’d like to discuss in more depth for the remainder of the meeting.

This is a common format for many Agile retrospective meetings. If this sounds all too familiar and you’d like to zhoosh it up a little, I invite you to peruse FunRetrospectives — after reading this, of course!

In every single team I’ve worked in over the last three years, I can hand on my heart say there has been at least one, often more meetings like I described where the issue of “Should we be doing Scrum or Kanban” has cropped up. It’s usually followed by prolonged periods of debate in which a consensus is rarely reached. It’s a debate I try and avoid, I’m old enough and daft enough to know that both processes have a part to play in software development. Both offer different things to different people, and both can be applied at different stages of a project’s lifecycle.

So Why Does This Conversation Continually Happen?

Reflecting on why this conversation occurs repeatedly has left me drawing a bit of a blank as to the root cause. However, I believe there could be multiple factors at play, which I’ll now discuss.

Project age: I’ve been a part of projects which have run for multiple years working to a regular release cycle — aka Scrum style. In addition, I’ve worked on projects which are in their infancy releasing ad hoc — Kanban. Both have had several debates over which process is better suited. So in my opinion, the project age doesn’t really come into it. However, there may be times where one process is better suited over the other, as you’ll see.

Team composition: I’ve witnessed teams of seasoned engineers and teams with mixed levels of experience arguing over Scrum versus Kanban. More recently, I’ve been part of a newly formed team consisting of mainly younger engineers exploring which is the right process for them. I have found that it’s usually developers/engineers who feel most passionate about this topic. Very occasionally someone from a different role will get involved. But more often than not, it’s the developers forcing the issue.

So why do developers feel so strongly about this issue? One of my friends recently noted that Scrum, with its time-boxed delivery and close ties with Agile methodologies, can be (mis)used to put pressure on development teams — in essence, making them feel self-conscious about their pace of delivery and putting the focus on engineering. This can lead to a feeling of being in the wrong or being monitored. Thus, creativity and quality start to slip in place of speed. You can have both.

Team leadership: Drawing on my friend’s observation about pressure, this is where I feel we may find some answers. The leadership of the team (by which I mean the roles responsible for setting the priorities) tend to have more influence over the processes used. In one of my previous roles, there were two analysts and one project manager. These were all senior members of the team and carried a lot of influence. The engineers often didn’t have the courage to go against them, or couldn’t be bothered? That team ended up being mandated to use Scrum despite murmurings from engineers around the coffee machine of how they’d prefer to use Kanban.

I’ve also been part of a team where the leadership consisted of a technical lead, project manager, and product manager. We used Scrum without any internal conflict. I believe this was down to a shared mission that everyone believed in, meaning the delivery process took a back seat. Everyone wanted to deliver constant customer value and had more pressing issues to solve than arguing over the process.

Organisation culture and reporting lines: Following the leadership trail upstream, you start to see who the key influencers are that surround the development team. These are the project stakeholders — the ones on the receiving end of output reports. Usually, these are middle-management types, who at some point in their career were at the coalface doing the technical tasks but have long since graduated to a world of spreadsheets, roadmaps, slide decks, and graphs — as I once did.

These people influence the engineering team indirectly to a great extent. I’m referring to roles such as delivery/project managers, planners, product managers, heads of engineering, and perhaps even your CTO. These are roles that play an integral part in the delivery of software but not the development. I’m talking from experience here, so do bear with me.

My theory is that SCRUM provides these stakeholders with a feeling of safety

Scrum is easy to understand, offers a degree of predictability, and can be easily translated to people higher up the chain of command or outside of the development team.

Take the following concepts for example: Sprinting to a guaranteed delivery goal every few weeks. Burndown charts tracking the delivery of work. Daily stand-up meetings that anyone can drop-in on, and with ceremonies such as planning and retrospective reflection — what’s not to like? It’s a simple sell and sounds like it could be easily applied to yield results. On the surface, it loosely aligns with the old ideals of command-and-control leadership, which most people default to.

Kanban, by comparison, appears much more uncontrolled and unpredictable. There are no sprints — you pull work in when there’s capacity, often leaving team members with slack to help others, tidy up, or learn new things. You limit work in progress to create flow in your development pipeline. You eliminate waste where possible. There’s no emphasis on estimating or story-pointing work. Work is tracked retrospectively using cycle times (days, not made up numbers), and it’s delivered in small batches.

Ultimately, there’s no fanfare every few weeks to mark a new release — no marketing moment. Releases happen when needed. This offers the engineers a greater degree of flexibility, as they’re not rushing to meet a sprint goal or feeling pressured to release before the end of a sprint — meaning work can ship when it’s ready.

Other Contributing Factors

Control: Reading between the lines, Scrum offers not only safety but control. There’s lots of time to proceed with caution to avoid potentially rash decisions because usually work will be planned within an inch of its life.

Backlogs will be built, refined, and pruned before completed work waits in a preproduction environment prior to being released. This gives ample opportunity to be absolutely certain the work is of high quality and poses no risk to customers or the brand. The pause before a release is a safety net people averse to risk flock towards in great numbers.

While you’ll likely have a backlog of changes with a Kanban system, these changes tend to be shipped when ready. If done correctly, there should be a steady stream of small changes making their way to production on a daily basis with a constant focus on prioritising the high-value items.

If bugs make it to production, the team should stop and prioritise fixing the issue before releasing more changes. Facebook famously follows the mantra of “the next release will fix the bug.” This puts the emphasis on speed and quality — the trade-off being a slight increase in risk. This may not sit well with stakeholders, who feel any production issues risk brand damage.

Branding: When it comes to Scrum, I feel people become biased because they fall for the branding. Just think if you weren’t the experienced software craftsperson you are and some consultant said:

“We need to go slower in order to go faster by limiting our work in progress”

What would your reaction be? I kid you not — I’ve been on the receiving end of a CEO scoffing at that logic.

On the other hand, if our consultant was claiming that we’d ship reliably every fortnight by working in well-planned sprints while expertly guided by a Scrum master, doesn’t that sound more appealing than these slow-coach Kanban advocates? Thinking about that enforced delivery mentality I touched on earlier, it’s easy to see why this sales pitch sounds appealing.

Project fit: Experience has also shown me that there are situations when SCRUM may suit your needs better than Kanban.

I’ve found the concept of sprints or timeboxing work fits well when a product is in its infancy. It’s important to gather feedback and demonstrate progress rapidly when building a new product, so sprinting in the early stages allows a team to deliver just enough before a review with stakeholders — in essence, when the need to measure progress is at its highest. That said, Kanban also works well here, but if there’s no forcing function such as sprints, teams can be left to drift.

Postlaunch Scrum tends to lead to many changes shipping at once in a batch. If your team is shipping once or twice per month that’s potentially a lot of change. This can be considered risky, both in terms of the changes all working and your customers actually appreciating the large swath of changes to their beloved product. Changes should, in my opinion, go unnoticed.

Contrary to that, I’ve worked on teams where shipping a batch of changes every fortnight worked really well. Everyone in the business was on a fortnightly cadence of show and tell, retro, then planning. The main difference was a shared goal and passion for the work we were delivering. Everyone on the team used the product too, and the leadership was inspiring — a combination that’s hard to find!

Another factor in your delivery-process decision may well be your infrastructure. It may not be feasible to deploy multiple releases per week or per day. Therefore, a slower delivery frequency may work well in your team. Your deployments may need some manual intervention. You may have to have an operations team deploy your changes, so batching them up makes sense.

In a previous role, we had the concept of a maintenance window — meaning customers were notified that problems could occur in this timeframe. I’ve also seen changes shipped in large batches out of hours when no one is using the system. Again, the trade-off here was a risk to the business that deployments might impact business functionality. This is where Scrum worked well.

I’ve found Kanban tends to work best when there’s already an established product and the team has switched to experimentation mode, trialing changes to see what adds value to customers — when there aren’t any hard deadlines and the backlog is mainly full of ideas and support tickets. This is where each development task can be pulled in, delivered, and then validated individually. This phase usually follows a major release, where incremental additions to the product are added. There are no big-bang releases planned, and customers will have to look hard to notice the incremental changes being made.

Prelaunch, Kanban can work too. It ensures engineers have the time and space for spiking work and determining if a particular design is going to survive in production. However, prelaunch is where that need to measure progress and work to deadlines is highest and trumps unpredictability. It’s this need that I feel drives a lot of teams to adopt Scrum in a project’s infancy, given its focus on regimental delivery.

So What Should I Do If I Find Myself in the Scrum or Kanban Debate?

Quote The Rock: “It doesn’t matter whether the team is using Scrum or Kanban as their framework for project delivery.” (I’m sure he said that on “SmackDown” or “RAW” at least once?)

What counts is that the team is delivering valuable work in a timely fashion to their customer(s) — period. The how is unimportant.

As I’ve discussed in other articles, outside of the development team, these decisions have little or no bearing on the customer. That’s why I subscribe to the Amazon principle of customer obsession. Everything a team does should benefit the customer. I feel development teams often lose sight of their customers because they’re abstracted away with roles like business analyst and product owner etc. taking the place of the customer.

Teams can fall into the trap of thinking just because they’re serving up an internal API they don’t have customers — wrong! Your colleagues become your customers. Those API’s, albeit not public-facing, should be designed with externalisation in mind. Why provide a crappy service to your colleagues and a better one to customers? Just make everything awesome, right?

If the product is public-facing, include customers as part of every decision or experiment. Make sure the whole team talks to customers, be it through feedback sessions, UX research, or, at the very least, surveys.

Something I feel is crucial here — and I feel corny saying it — is: These debates are often happening within teams and organisations purportedly claiming to be Agile. But if teams are arguing over which process to use to manage and track work, you’ve failed the first principle of being Agile which is …

“People and Interactions over Processes and Tools.”

Surely, if you’re an Agile organisation or team. you shouldn’t be having these conversations. Work coming into the teams should be originating from that customer collaboration. You should be liaising often with your customers to regularly demonstrate your working software. How the team tracks this work doesn’t deserve scrutiny. There shouldn’t be weeks of sprints planned and milestones to hit. You should be responding to the needs of your customer.

Scrum and Kanban are processes. True agility comes from listening to your customers and conducting experiments to find that product-market fit — not having three hour planning sessions and retrospectives that cover the same ground every fortnight.

One day I want to see a wall of work with the following columns:

I’m not suggesting we all walk into the office tomorrow and tear all the Post-It Notes off the walls in fits of rage — more that we should try to realign our focus away from these internal debates and redirect that energy towards our customers’ needs.

Have the whole team in conversations with customers and not just the UX team. Product owners can yield real breakthroughs in their thinking by observing customers interacting with the products they’re building — so try to do more of that and less internal squabbling over processes!

I appreciate these are very brief overviews and very personal to my experience — for more information please see:

• Scrum
• What is Kanban?

Scrum vs. Kanban and How to Stop Fighting About it was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.