Healthy Backlogs
Co-author: Anuj Desai
Anuj and I have had the privilege of working with several teams on their roadmaps and sprint plans over the last few years, and we noticed that it is not always obvious what a healthy backlog should look like, or how it can be maintained. Unhealthy backlogs, on the other hand, are much more apparent.
Let’s declare <X> bankruptcy.
- Each of us, at some point in our careers, where X is often email, backlog, and sometimes PagerDuty.
Too Long
The foremost example of an unhealthy backlog is one that grows in length uncontrollably. Week after week, more than twice as many tickets are added to the backlog, compared to the number of tickets that are resolved or closed.
Often, this accumulation festers a feeling of helplessness on the team. There could be rampant pessimism and defeatism: some engineers might even be close to burnout or despair as they try to churn through as many one-off requests as they can. Adding new items to the backlog feels pointless and insignificant. Reorganizing and pruning the backlog becomes arduous. As a result, tasks get tacked onto the front or tail-end with no relative prioritization.
Over time, this manifests as “Nobody cares about X”, where X ∈ {quality, performance, operations, security}. Stakeholders start to wonder if anything will ever get done, and the team falls into a downward spiral of urgent interrupts. Since the team can never get to most of their maintenance tickets, systems decay. When the next customer escalation or outage happens, the only sane thing they can do is reach for duct tape, which reinforces the spiral.
There is just no time to do anything else! This team needs a turnaround.
Borrowing from The Progress Principle, the first step is to find small wins to rescue morale. Start by negotiating with stakeholders to set new expectations. Then, set aside time for maintenance and automation improvements on a single narrow area that reduces toil and saves time. It is crucial to maintain focus until the team can see the fruits of their labor; let observers know that this is the only thing the team is working on. As the team’s leader, take the time to share results, report progress, and start rebuilding trust.
Once these efforts bear fruit, move on to the second narrow area, compounding the productivity gains from each project. At each point, limit the amount of work in progress to mitigate the risk of falling back into the downward spiral. Eventually, the team should feel that they can come up to the surface for air, and morale should show a visible lift.
Next, rally the team together to set goals for themselves for the next 3 months. Prune them by selecting for projects that contribute toward a few key results, which align with the company’s objectives. Update expectations with stakeholders that everything else has to be dropped, or even removed from the scope of the team. Lean on the trust and confidence the team has built in the turnaround to garner support.
Finally, using these goals as guidance, do a one-off clean-up to prune all tasks on the backlog that do not contribute to these goals. This is a fresh start.
Too Short
On the flip side, a backlog can be too short if it is frequently emptied. Each week, more tickets are removed compared to new tickets filed. This results in an empty list, or a list that is populated with nice-to-haves, instead of tasks that are necessary and important.
Over time, one may find engineers prototyping with technology that has a tenuous link to the company’s business. Do these sound familiar?
- Using Kafka everywhere when there are no real requirements for real-time processing.
- Building strange apps on the blockchain because they have to be distributed for some reason.
- Migrating to Kubernetes because “that’s what the industry is trending towards”.
Even relevant projects start getting more and more over-engineered with odd features that stakeholders did not ask for and will not use. This team needs a vision and a plan. More experienced engineers will question: what is next, and why are we building our own database again?
These are signs of growing internal complacency, and the biggest challenge is to create a sense of urgency. This team needs realignment.
Go to your stakeholders and have a conversation to gain a sense of when you can expect the backlog to grow for the artifacts owned by your team currently. Next, adapt to the company’s highest priorities and help your team expand its scope. Take advantage of the available headroom to find impactful projects that materially strengthens the business. As scope grows, continue building relationships with stakeholders and create a feedback loop.
Once the relationships have been cemented, use the feedback loop to contribute ideas, tasks, and bugs to the backlog. Your team has the most context on where investment is urgently needed to reduce maintenance burden and manual toil. All ideas for future improvements should be discussed and prioritized to avoid situations where an individual uses the time to build something that is not meaningful to the business.
Just Right
A backlog that is “just right” should feel like this:
- There is constant pressure to make trade-offs and prioritize, along with an understanding that we can’t do everything. If we are spending time on investigations and futuristic prototypes, there must be some concrete thing we are giving up, such as an important (but hopefully not urgent) maintenance task.
- There should be a good idea of what is next. The trade-offs and prioritization lead to a plan. We know we have work to do, and here is what we are going to work on immediately, and what we are going to defer. There is consensus on what we have decided to drop.
- The team knows where to find the backlog, how to manage it, and when it is too getting too long or short. We may still have some team members that feel that the backlog is just too short if they don’t know where it is or some that think it is too long if they are creating tickets that get lost in the abyss. It is important to align with the team on what a healthy backlog looks like and how we will track it.
- The team feels energized and is aware of the impact they are making on the business. As a result, such a team is also constantly aware of the impact of each trade-off.
Aside: Pruning Aggressively
To keep the backlog healthy and manageable, it is important to prune aggressively. This includes:
- Moving to icebox: tickets that are not ready to be assigned, perhaps what is required is too far of a technological leap for now (this corresponds to the “Someday/Maybe” list from GTD).
- Closing as Won’t Fix: this communicates to the ticket reporter that the request does not align with the team’s vision or is otherwise not feasible in the existing architecture.
Pruning is easier if you categorize your tickets effectively in your backlog. They will be easier to sift through than a giant list of miscellaneous items.
It takes constant attention to curate a healthy pipeline of work that will push your team to deliver impactful work at a sustainable pace. A healthy backlog translates the teams’ goals to day-to-day tasks and contributes toward business needs.
How do you evaluate the health of your teams’ backlogs? Tell us about your rules and heuristics.
References
- Backlog Management Index: Kan, S. H. (2003). Software Quality Metrics Overview. In Metrics and models in software quality engineering. Boston: Addison-Wesley.
- Amabile, T., & Kramer, S. (2011). The progress principle: Using small wins to ignite joy, engagement, and creativity at work. Boston, MA: Harvard Business Review Press.
