Hi all!
Today's post is about software backlogs, why they often turn into quagmires, how to fix them, and better yet, how to prevent quagmires in the first place. Also we'll dunk a little on JIRA, you know, because we can. (Every word of ifuckinghatejira.com is true.) The most important tip is that I try to do as much story writing, collaborating, and revising in a shared online document before the stories are stable enough to load into a ticketing tool, heading off a mess, but there's more to it, so read on!
The Ideal State: A Nice Pipeline of Work
In almost every book on agile I've ever read, they describe a beautiful flow of work through the backlog. Large, partially formed pieces are at the bottom of the list, then they get refined and broken down, and finally, the small pieces that are ready for engineering float to the top, where they can be loaded into a Sprint for engineers to work on.
Periodically sorting and cleaning the backlog (often called Grooming) is meant to keep the backlog in a nice, neat, prioritized state, with the highest priority, most important work always at top.
Engineering is often the most expensive, labor intensive bottleneck in a product team, so keeping a steady supply of the highest priority work ready for the engineers is meant to ensure the team is working on the optimal thing.
In theory, this sounds lovely, but then real life happens, and you get a mess.
The Problem: Thousand Item Backlogs
When I say mess, the problem that I've repeatedly seen is software teams with hundreds and even thousands of items in their backlogs. Nobody can hold that much stuff in their head, and I'm betting that the "prioritization" in the backlogs just means that the most recent items with the most attention end up at the top of the list. The queue starts to form from the front, and the rest of the backlog is dead. It's hard to having meaningful grooming discussions over what's important.
How did this happen? The analysis:
Having better knowledge of story writing and a good process for breaking down stories certainly helps, but let's be clear here: most PM tools set you up to shoot yourself in the foot.
The first thing that happens, is that the act of writing down the story into a ticket in any one of these tools makes it inflexible. Creating tickets, naming them, filling in fields, - all these acts create user friction that makes it slow to split, change, or modify a story later on. (JIRA, somehow, even marks it hard to delete a ticket, rather than archiving it or progressing it through the usual workflow.)
The other thing that happens is that the text field of the ticket itself is a terrible text editor for collaborative work. Any comments on the ticket are placed as footnotes, rather than inline, and any of the features of a modern, collaborative online editor are lost.
So what happens, is that the stories, once created, end up looking something like the picture below, and they never get refined.
Some classic symptoms of this failure mode are stories that take forever to close, lots of work happening outside the ticket system entirely, and piled up and abandoned stories that were no longer relevant after they were written.
More Analysis: Epics, Stories, and Tasks
The other place I've seen tools lead people astray is by creating the illusion that work is being smoothly broken down, providing ticket types like epics, stories, and tasks. (And in the case of JIRA, also sub-tasks. Again, please see ifuckinghatejira.com)
So in theory, you should be able to create epics, then break those down into user stories, than in turn, detail the direct work or "tasks" that go into each user story. (Others may use different pieces of terminology, I prefer epics, stories, and tasks).
Once again, real life happens, and the shortcomings of the tools become apparent. JIRA in particular, has UI quirks around how it implements sub-tasks and if they even appear in a backlog or on a scrum board. (Trello, Asana, Clickup, and others all handle subtasks much better.) JIRA quirks aside, the tools now fail in a handful of ways. Either all stories are visible in the backlog, regardless of refinement level, or the backlog is limited to a single epic, regardless of whether the team has multiple epics open.
When stories from multiple epics coexist in the backlog, I usually say we've gone from "apples to oranges" comparisons, to comparing "apples, oranges, raisins, walnuts, and bananas." It just doesn't work. Prioritization needs to happen at a higher level.
The Solution
The solution is to two part. First, maintain separate backlogs at multiple resolutions. Second, keep the fine resolution refinements in a collaborative document and out of the ticketing system for as long as possible.
Typically, this means I'll write out user stories and acceptance criteria in a Google Doc where they can be commented on, debated, easily rewritten, or split. When the stories are stabilized, and we're to the point of understand the work tasks, I'll load the stories into JIRA, where they're ready to be estimated, prioritized, and scheduled into Sprints.
Finally, for the fine resolution backlog, build filters so that only the very top of the "coarse" resolution backlog appears in the fine grained backlog.
What this ends up looking like in JIRA is a scrum board for Epics and a scrum board for Stories, with some filtering to determine which epics are allowed to appear on the story board.
Wrap up
I'm shocked at how few tools support flows like this, and in particular, how tricky it is to implement in JIRA. For my money, shortcut.com is probably the best "out of the box" tool I've seen for software.
There's also no substitute for having a good sense for what a "story" is, as useful, functioning piece of software. (I still high recommend User Story Mapping) and a process for designerings and engineers to break down stories together, but that's a topic for another blog post.
Need help? Want to talk more?
If your team could use help getting your backlog in order, I'd love to help. Just email me!
best regards
Sam Feller
aka THE Awkward Engineer