Prioritization, multiple work streams, unplanned work. Oh my!
On development teams, balancing priorities while keeping focus is hard. Without an intentional approach you can quickly find your team submerged in a lot of different work, hurting progress on your most important goals.
But the real world is messy. Prioritization is hard. Work comes in many shapes and sizes. Unplanned work pops up at the worst times.
Managers face many difficult questions:
Should you have multiple work streams on your team? If so, how many? How do you balance long and short-term priorities? How should you handle unplanned work? What about on-call? How big should your team be?
This post outlines some approaches I’ve found effective in helping my teams maintain focus and get stuff done, amidst all the chaos and unpredictability of the real-world.
First Principles
It’s important to start with some first principles for doing any kind of work. These principles form a foundation for any sensible approach to predictably completing work.
When everything is important then nothing is important
There must always be a priority stack rank to all work. Full stop. You have to break ties. That means allocating people to work based on priority first, not effort. There needs to be a crystal clear understanding of priority when multiple pieces of work are involved or your team won’t be able to make effective decisions in real-time, and it will be left to happenstance. When multiple things are of equal priority, everything thrashes.
Prioritization works even more effectively when there is broader prioritization to categories of work. For example, “customer trust” being a higher priority than product roadmap work. This allows teams to quickly make important prioritization decisions about high priority bugs or issues that affect customers without unnecessary friction. This is empowerment!
Finish what you start
Optimize for finishing work before starting new work. All other things being equal, when you introduce a new stream of work, you slow down existing work. The same people are now spread across more things. Everything suffers including the new stream because it isn’t being set up for success, given that nothing else has stopped. If you do decide to stop something else then all momentum on it is lost; along with a potential hit to team morale. It’s a lose-lose situation.
Optimize for finishing work before starting new work.
There are exceptional circumstances where you need to stop or starve an existing stream of work in order to start something new for an urgent or unexpected reason. This should be rare and for a really good reason, based on external factors outside of your control. There is no other good reason to do this. If it is a common issue for you and your team, there is something else going wrong. Contributing factors might be rapidly changing priorities, organizational politics, or a misunderstanding of the software development process.
Optimize for reducing uncertainty
This is project management 101: de-risk things early. Enumerate your unknowns and tackle them in priority order based on level of risk. As you progress through a project, risk should strictly decrease, with the largest reductions occurring at the earliest phases of the project.
Expect the unexpected
Just like failure in software systems, setbacks are inevitable in all projects. External disruptions, sick days, on-call fires, dependency issues… are business as usual; they’re normal. Think of them that way. The occasional surprise or setback should not derail your project. If it does, you have failed to plan for the unexpected. Planning should reflect an unplanned work allocation¹. Expect the unexpected, and build extra time into your plans so you can improve predictability. This will also make work less stressful for you and your team and go a long way to building trust with leadership in your organization.
The Pareto principle
Internalize the Pareto principle, also known as the 80/20 rule. It is ubiquitous. 80% of the effort is spent on 20% of the work. 80% of the planning is done by 20% of the team. 80% of the risk in a project is in the first 20% of the project. You get the idea.
The Pareto principle is a very useful way to think about how you approach work generally; helping you think beyond a pure headcount mentality when assigning people to work. What requires more thought? More planning? What needs to be more resilient to disruption?
The Pareto principle can also inform decisions on who works on what. Don’t fall into the trap of thinking that spreading people across work will lead to more progress. It will only lead to more thrash disguised as activity.
Rules are made to be broken
Remember that rules are made to be broken. With any project management practice, its important to remain flexible and use guidelines as guidelines, not hard and fast rules. Be pragmatic, flexible and prepared to bend the rules from time to time. But don’t use rule bending as an excuse to drop process entirely. Even good process can lead to poor results when followed blindly. Decisions should always be made within the appropriate context. In other words, use your intuition and common sense.
Work stream best practices
Now that we have some first principles, let’s look at some best practices for managing streams of work on your team.
Assign engineers to streams of work
Context-switching is expensive. Keep engineers on the same stream of work whenever possible. Don’t bounce engineers between streams. Context switching between tasks is expensive, but context-switching between work streams is even worse. A work stream has an entire surrounding context beyond the task at hand, which must be brought into mental focus when a switch is made. Context-switching between streams kills momentum. When someone stays on the same stream they can build momentum and are more likely to smoothly transition from one task to the next and maintain flow.
With continuity, an engineer is also more likely to take ownership over the work. This leads to more engagement and the opportunity to proactively anticipate or notice problems in discussions, code reviews, and suggest improvements. Finally, people just don’t like context-switching. Do you enjoy hard-switching tasks? Just when you build context for one stream of work, you are torn away to switch to another. It’s disorienting.
Prefer a single stream of work when possible. 2 max.
In an ideal world, you would have a single project that is your highest priority and it would be a team-sized effort. With the whole team focused on this one project, you would have total focus, finish faster and be the most resilient to unexpected problems.
In reality, this is rarely the case. But while it may not be true for a project from start to finish, it is frequently true for stretches of a project. In these situations, you can opportunistically pull in smaller pieces of work to maintain a largely singular focus over a period of time. This approach helps mitigate crowding, or the “too many cooks in the kitchen” problem. This causes thrashing to happen in a single stream of work due to insufficient surface area for the work or work available. This is very common in the early phases of a project, for example when infrastructure is being provisioned or there is a sequence of relatively serial work that unlocks work that can be done in parallel later on.
It also helps build trust and patience with your product peers and senior leadership on longer projects by producing smaller deliverables that make customers or stakeholders happy in the near-term. I’ve been able to do this successfully on a number of long projects.
Max 2
Given that one stream might not always work, how many is too many?
I recommend instituting a policy of having a maximum of 2 work streams at any given time. Why 2? In my experience a reasonably sized team (6–8 engineers) will start to thrash at 3 streams of work; The team starts to spread thin across the work and streams are easily derailed by unexpected issues that inevitably come up.
The work also begins to skew to more of an individual-oriented effort, rather than team-oriented effort, which weakens unity and team morale. However, you do need a minimum team size to support this. In my experience, at least 6–7 engineers are needed to have 2 productive and substantive streams of work.
2 streams should always be treated as major and minor. One is always larger than the other. Typically the larger one is the higher priority. Never do 2 big things at the same time. One big and one small is a good rule of thumb.
Never do 2 big things at the same time. One big and one small is a good rule of thumb.
A caveat to the max 2 rule
There is one caveat to the max 2 rule: during the transition period between two projects.
As one thing is winding down another can be starting up, typically entering a planning or work breakdown stage. The key to making the transition work is that when an engineer rolls off the first project they never return to it. They move on completely to the next priority or project.
So who should transition off?
As mentioned above, typically the next project needs planning & work breakdown, so a project lead is a great choice. The lead transitions off to begin planning so that when the engineers on the prior stream complete it, everything will be ready for the team to kickoff execution of the new project.
An engineer moving from one in-flight project to a new one can make sense in a few other situations as well. For example, a project nearing completion and no longer needing as many people to finish it. This isn’t usually a problem, since there are typically opportunities to help with planning, do personal development or a variety of other activities that are valuable for the team and the individual.
Streams and engineer engagement & growth
If a work stream is long-lived, another consideration is whether an engineer should stay in that stream long-term. While projects in different streams may not complete in lockstep, there are usually opportunities to provide optionality to engineers to move across streams — say from major to minor or vice versa.
People like variety, and it comes in many forms: bugs vs features, big vs. small projects, project leadership vs heads down coding, and being in or out of comfort zones (e.g. stretch assignments). All these experiences are important for engineers to stay engaged and avoid burnout, and streams can be a useful mechanism to provide this variety. Some prudent planning and thinking ahead is very important and a key role for the manager to play.
The on-call stream
There’s an ebb and flow of interruptions that naturally occur on teams, especially when a team owns and operates the services they build.
A special case stream of work I found works really well over the years is the idea of a special mini stream: the on-call stream. This is not counted in the max 2 rule (yes I know I’m cheating here). This is a work stream of serial work items. Only one ticket can be worked on at a time in this stream, and its worked on by the current on-call engineer in the rotation.
The idea is simple. During an engineer’s on call shift, they do not work on the team’s project work. Instead they focus on interrupt-driven work and opportunistically tackle unplanned work such as tech debt, critical bugs, security issues and the like. In terms of the opportunistic work, a framework I have found works well is 50% customer issues / 50% tech debt.
On-call stream benefits
Having an on call stream provides several benefits:
- It makes project delivery more predictable. You simply don’t consider a single (rotating) team member to be allocated to projects during planning. This means during a week that is particular interrupt-heavy there is less risk of project work being disrupted. It also means the on-call engineer can focus on just interrupt work, not having to juggle the stressful experience of trying to make progress on ticket work while fire-fighting and being constantly interrupted.
- There is something for everybody. Product managers love it because bugs don’t languish in the backlog, and high priority customer issues get attention quickly; Engineers love it because they get to be proactive about tech debt² and maintain better hygiene in the code and services.
On-call stream tradeoffs and challenges
Having a full on-call stream doesn’t make sense for every team and every situation. It depends on several factors including your team’s on-call load, the number of services they own and the size of the bug backlog.
Every time I have introduced the on-call stream to a team, I have first done so as an experiment. That has proven to be an effective way to discover if its a good fit for the team, as well as involve the team in the decision and the implementation details.
The on-call stream is not without its challenges, however. You have to be mindful of how you handle work spillover on either side of an on call shift; either unfinished project tickets spilling into a shift or an on-call stream ticket spilling over the shift back into project work.
Ultimately, taking a pragmatic approach works best. You have to be especially mindful of ticket size and not pick up a larger ticket towards the end of the week. My general rule of thumb is a 1-day spillover on either side is fine, but if you think a ticket will take longer than that, you should look to hand it off or bring it to a logical stopping point if that’s possible. As with most things, there is the 80/20 rule (Pareto again), exceptions are sometimes warranted in important situations.
Closing Thoughts
For all the considerations and approaches outlined in this post, it’s important to evaluate them not just for different teams, but also for the same team over time. Retrospect & adapt. Teams are always evolving, and what worked 6 months ago may no longer work today.
If you find there are consistently two streams of work each belonging to a thematic area, that may be a signal that it’s time to consider splitting your team. You may be missing an opportunity to create a more cohesive mission or charter around one stream and create additional leverage and growth for the business and the engineers. Of course, this should align with the needs and investment appetite of the business for that particular area.
Conclusion
Managing priorities and organizing a team’s work can be challenging. It’s helpful to have a framework to guide decisions on when to split your energy across a few priorities or have total team focus. No team or situation is exactly the same. Weigh the pros & cons of which approach you take based on the size of your team, the nature of the work, and your overall support burden. Good luck!
[1] This is not the same as estimation buffers. Those relate to estimation accuracy, not external factors.
[2] This should not be the only means of tackling tech debt! See my post on the risk matrix for more details on a process to integrate larger tech debt items into your actual roadmap.
Originally published at https://leeorengel.com on January 24, 2022.
