2.2 Roadmaps and OKRs
2.2 Roadmaps and OKRs
We have to empirically measure progress toward success. "Features delivered" is old hat. Create customer-facing value every single time by knowing what to measure and how to keep measuring.
Writing this chapter is a little challenging. Even intimidating. I needed to find a way to turn something that’s a bit bothersome, even tedious, into something meaningful and hopefully engaging. Because it’s emphatically important — yes, I know I’ve said that about a lot of these early-stage activities. The success of your project hinges on establishing a solid foundation. So here goes. Let me know how I did in the comments.
Introduction
This chapter is about measuring success. Specifically, it’s about turning the objectives you’ve chosen for your project into action that can be measured. The bit about measuring is particularly important because if you can’t measure well, it’s inevitable your project will fail to some extent or another.
That’s why this chapter is so important. If we don’t set ourselves up now for success, we will not succeed.
Picture this: You’ve agreed on an overall strategy for your project. The business objective is to lower operational cost. There’s a second objective to lower customer cost as well. The reasoning is, you’ll do so by passing on operational savings to your customers. As a strategic goal this is excellent but without a system in place to execute the idea nobody will know where start. Likewise, without agreement of how to measure success (and exactly what “success” looks like), there’s no way to know when or if you achieve significant results.
There are two paths forward: Without measuring, and with measuring. Let’s explore these scenarios.
Without measuring
We introduced some objectives. Let’s state them a little more clearly:
Lower operational cost by reducing poorly utilized infrastructure and improving the application operational characteristics.
Reduce the product price for customers by passing on those savings.
From here, you might choose to launch directly into implementation. There may be obvious places to cut costs — perhaps consolidating some server infrastructure, turning off unused or under-capacity systems. That’s probably the low hanging fruit, relatively easy to identify and implement. But we can go further. Perhaps the next step is to terminate an expensive Oracle database license in favor of an open source solution like Postgres. That’s going to take more effort, but thanks to Postgres’ relatively high level of Oracle compatibility it might not be too difficult. Beyond that, you can embark on some serious application rearchitecture and migrate to a highly distributed, scalable model — possibly embracing CQRS and actor sharding along the way, squeezing every little bit of distributed power out of a small cluster of server nodes. That, of course, is a lot more work.
Depending on your affinity for this kind of work, it can be exciting! A whole new architecture using the latest, forward thinking in distributed systems. Pretty neat. Also potentially expensive to build.
So what happens when leadership cracks down, demanding to know why your engineering project is spending more money than it’s saving? When they ask why important features are being delayed for this “cost reduction” project — one that isn’t actually generating the kinds of cost savings originally envisioned?
With measuring
There’s a better approach, one that establishes clear and specific outcomes and measures progress toward those outcomes. The goal is to make sure your objectives are achievable, that the team stays tightly on-track toward those goals, and at the end of the project you’re in a position to say “mission accomplished.” Or better, “mission exceeded.”
That’s what this chapter is all about. It’s the difference between declaring victory every time versus having fuzzy, unclear goals that miss the mark in unexpected and sometimes very disappointing ways.
That’s where roadmaps and OKRs, or objectives and key results, come into the picture.
What are OKRs (objectives and key results)?
Stated simply, OKRs are a framework you can use to define, coordinate, and accomplish goals in a demonstrable way. “In a demonstrable way” is the differentiator here. OKRs will set us up to empirically know that we are on track, allow us to course-correct if we get off track, and give us the ability to declare “mission accomplished” with cold, hard facts.
Here’s a simple definition for OKRs, courtesy of Forbes:1
Objectives are specific and clearly defined goals that will have a major impact on the business. They should be challenging but attainable, and align with an organization’s strategic goals.
Key results are how an objective’s progress is measured or monitored on the path to achieving the goal. Key results serve as outcomes that provide evidence of whether a goal has been reached or an objective has been accomplished.
We might write an OKR like this:
We will reduce our operational cost by 30% as measured by the month-over-month spend of our AWS infrastructure.
A very little history
According to Forbes, the methodology for OKRs was developed in the 1970s by Andrew Grove, Intel’s CEO. Grove taught one of the company’s most successful salespeople, John Doerr, about the concept. Later, Doerr served on the board of Google, introducing its founders, Larry Page and Sergey Brin, to OKRs. In 1999 Google implemented OKRs as a business practice and has used them ever since to great effect.
In the previous section 2.1 Product strategy, we established our business objectives. These are what we’ll be working with. Your business objectives may already be clearly stated goals — or, they may need a little refining. That’s fine, this is our opportunity to make sure every objective has a measurable outcome.
Conceptually this is pretty simple — just establish measurable outcomes for clear goals. But, there’s a hard part too: Deciding exactly what to measure and then ensuring we never lose sight of those measurements or stop measuring.
What to measure
Sometimes it’s hard to figure out exactly what to measure. In our example, we set the business goals on reducing cost of infrastructure. That leads to a nearly obvious, measurable goal of cutting our month-over-month cost. Easy peasy, as the saying goes.
But what about more nuanced situations? For instance:
How do you measure how quickly the team gets new features from concept into the customer’s hands?
Can you measure regulatory compliance?
Is it possible to measure an increase in revenue as a leading indicator, distilling measurable metrics at the development phase?
What about measuring customer satisfaction in a meaningful, clear, fact-driven way?
In my experience, everything can be measured with metrics you can test. Sometimes it’s easy to figure out what those metrics are — and sometimes it takes a bit of brain power, collaboration, and agreement on what “moving the needle” actually looks like. That’s where having a system that models your return on investment is crucial. It’s not acceptable to make simple assumptions, like, “if there are more users, they must be happier users.”
One tool I’ve applied in nearly every single situation is a mapping framework that distills business objectives to straight up revenue earned. I know this can sound a bit cold and capitalistic, but it’s the reality: Businesses operate by staying in business. Even non-profit businesses must make positive revenue or they will go out of business. Because of this, everything comes down to dollars — even, “let’s improve our customer experience.” That objective turns into happier customers that stick around longer, ultimately generating a positive impact on revenue.
So unless you are literally working for a philanthropic enterprise with the objective of giving away all of its money, this approach works. (And in that case, you can still connect outcomes to dollars — just the negative flow of dollars, not positive).
It can be helpful to have a map handy. The following is a map that traces most business objectives (such as improving cost efficiency) to specific strategies (such as avoiding cost), along with a definition of what that might look like:
Using a map like this can be quite helpful — it reminds us of how we can translate those high level business goals into tangible, measurable metrics that can be instrumented and monitored.
Never lose the “line of sight”
Another tricky part is making sure those original goals are always on everyone’s mind. I like to think of it this way: If I were to ask any engineer on the project, at any time, “why are you working on that?,” they would know:
Exactly how their work is going to affect key results,
Furthermore, they would know exactly how important their work is toward that goal, and be confident that there’s nothing more important they could be working on,
Finally, they would know exactly how value is going to be proven using simple, fact-based testing.
This awareness and confidence provides a line of sight from any point in the project back to the original business objectives and the value those objectives create.
The trick here is to create multiple checkpoints and feedback loops. That’s what the playbook does. In the following diagram you can see how major milestones and activities in the playbook constantly checkpoint your key results. We protect customer value by continuously re-injecting those key results into the delivery pipeline and insisting that the only path forward is a positive measure of outcomes.
If you use the referral button below, you’ll earn free premium access to Customer Obsessed Engineering. Just three referrals will earn a free month!
The benefits of empirical measurement
According to research from Asana, even though most companies set goals, only 26% of employees have a clear understanding of how their work impacts those goals. That’s because most teams set goals at the start, but then forget about them. On the other hand, when employees have clarity on how their work impacts company objectives their motivation doubles. By connecting individual work to organizational goals, employees understand why their work matters.
With that in mind, let’s get down to work. The rest of this chapter provides a specific approach to refining your business objectives and establishing clear key results. Those will then become the basis for our line of sight to customer value.
Before proceeding, let’s check that our definition of ready aligns us for success. It relies heavily on the outputs created in the previous activity on establishing product strategy:
If any of the above activities haven’t been finished, go back to 2.1 Product strategy and wrap them up. This will make sure we have all the information needed to proceed.
Guide to roadmaps and OKRs
In this activity, we’ll refine our business objectives where necessary and create our key results, stated as measurable outcomes. We’ll use the combined objectives and key results as the foundation for our roadmap moving forward. Just keep in mind:
Objectives are qualitative while key results must be measurable and quantitative.
Roadmaps are short-term and focus on specific outcomes (not tasks).
An OKR should be a clearly stated objective followed by at least one supporting key result. You can have as many key results as needed to achieve the objective, but having more than five to seven probably indicates your objective is too broad. You might want to refine it — breaking it into several different objectives. If we create objectives that are too big, it becomes hard to prioritize and deliver manageable increments of the system.
The key result is most effective when stated together with the objective, using a format such as, “we will [objective] as measured by [key result].” For example:
We will solve customer satisfaction issues with the web site as measured by 99 out of 100 people being able to complete transactions in less than 1 second with no errors.
What not to do (common mistakes)
Don’t mistake the actions taken to achieve an objective as the key result. For example, this is not a key result:
Objective: Reduce the number of data errors in the system.
Key result: Install release 10.0 of the web site software.
On the other hand, these are well-stated key results:
Objective: Reduce the number of data errors in the system.
Key results:
As measured by reducing the number of data quality errors reported to the support desk from 20 per day to no more than 1 per week,
As measured by reducing the number of orders that cannot be fulfilled automatically to less than 1% of all orders in a given day,
As measured by reducing the number of problems reported by customers trying to place an order to less than 3 per week.
OKRs and agile development
OKRs and agile go well together. In fact, agile becomes highly effective when combined with effective OKRs. Agile planning calls for regular check-ins on progress, re-evaluation of the current business context (changing business conditions, market conditions, customer opinions), and quickly adjusting to adapt to that intelligence. Having clear objectives and key results makes gathering that intelligence much easier.
What used to be a quarterly business review (asking questions like, “did we deliver the features we promised”) is old hat. Instead, we can measure more often, asking at the end of every sprint, “did we achieve the objectives we wanted?” This makes it possible to more rapidly decide what the best path forward looks like.
Matching objectives and key results
It’s important to find key results that are leading indicators of your objective, as opposed to lagging indicators. You want results that can be measured regularly and frequently, and lead to the overall objective. For example, if your objective is to improve recurring revenue (an annual outcome), you’ll likely want to measure quarterly revenue and subscriptions.
Our goals, therefore, are to establish clear objectives along with measurable key results. We need the results to be measurable using quantitative metrics. Our automated delivery pipeline means we need to set up metrics that can be measured automatically, in our delivery process.
Inputs
Clear, concrete business objectives that encompass the current delivery iteration.
The right people and team participation (e.g., Product Owner, business analyst, business domain experts, customer).
Process
Remember that the most effective way to establish measurable key objectives is to “look for the money.” This just means tracing the objective down to its business driver. That business driver is almost always going to have specific monetary goals. Use the ROI Map introduced above as a starting point.
Our goal is to define clear, measurable criteria for each objective. These will be stated as OKRs.
This is a workshop exercise, where everyone is expected to contribute. This is particularly important — it’s everyone’s opportunity to identify exactly what constitutes “success” in their mind. In some cases success is clear, but in others it can be an agreed-upon destination. For instance, “improve customer satisfaction” is an objective that can be achieved through different outcomes. This is the time to establish what those outcomes are.
By defining clear key results, we are making our strategy actionable. The key results become the basis for acceptance criteria in the development pipeline — automated, measurable tests that we can use to protect and align on customer value.
Step 1: Define OKRs
Refine business objectives to ensure they state measurable outcomes. If the team cannot agree on a specific outcome that indicates what success looks like, you may need to adjust your objective.
Trace objectives to quantitative, measurable outcomes. Use a system such as the ROI Map introduced above to stay focused on tangible outcomes for the business. While revenue does not have to be the success indicator, it is almost always a very strong indicator of success. You can use the relative amount of revenue as a simple way to prioritize your outcomes into a roadmap (below, in Step 2). In other words, the more revenue an outcome generates, the higher its priority.
Define metrics-driven, observable key results for each objective. State each of the key results that will achieve the business objective. Make sure that each key result is grounded in an observable, quantifiable metric. Use the format “we will [objective] as measured by [key result]” to define your OKRs.
Step 2: Establish your roadmap
As you create a complete picture of your business objectives and key results, you will find different results will yield more impressive outcomes. For any given objective, you might have half a dozen key results — but one or two of those will likely “move the needle” more than the others.
Build your roadmap by prioritizing the most impactful key results. These higher priority results will drive early initiatives the team focuses on. It’s important that a clear priority is assigned to each key result, so the team is aligned on where to focus early efforts.
The following example shows how you can group your OKRs by priority:
You’ll notice that we’ve added a “function,” as well as priority. This is simply a way of helping us group together similar functionality and identify which outcomes are the most important.
By grouping outcomes by function, we are building the basis of our roadmap. As your roadmap becomes more complete, you’ll see similar capabilities falling under the same function, or the same initiative. In the above example, “order management” is one such grouping — and it clearly represents the most impressive gains. It’s clear from this roadmap that we should tackle order management first, and data cleaning later.
The use of the word “function” to categorize outcomes is intentional. It coincides with the business functions specified in chapter 1.4 Business capabilities & functions. Your roadmap and OKRs will inform your business capabilities and functions, and vice-versa. Neither are fixed in stone. As new discoveries are made, adjust them to accommodate new learnings, further refine objectives, and improve key results. You can use this template as a starting point to build your key results and roadmap.
Outputs
OKRs. Quantitative results against which success is measured (e.g., “operating cost reduced by 50% as measured by month over month infrastructure cost”).
Roadmap (prioritized functions). An initial set of actions and initiatives that will support the OKRs, ordered by most impactful first.
You may have already started the 1.4 Business capabilities and functions activity, where your create an initial list of business capabilities. Now is the time to go back and make sure everything aligns. Your project wiki includes a section titled product vision, strategy & capabilities. You can now add fidelity to your Product strategy by:
Ensuring that the stated business capabilities and functions are clear and align with your final, refined business objectives. They should be clearly related.
Add prioritization to your use cases based on the roadmap you have defined.
Add the details developed in this activity to the wiki, further fleshing out your overall product strategy.
Templates
Next activity
This chapter of the Delivery Playbook is a free preview, but don’t miss out on the whole playbook. The complete playbook has over 32 full chapters, an equal number of supporting articles, project templates, and extensive references! All for a modest subscription that many employers will quite happily let you expense.
Publication details
Accessing the Customer Obsessed Delivery Playbook online
Paid subscribers can access the Customer Obsessed Delivery Playbook online, either from this link or by clicking ‘Playbook’ in the menu bar on the Customer Obsessed Engineering home page.
If you’re a free subscriber, you will have access to previews of most chapters, as well as select full chapters as they are released.
Can my organization get the complete Customer Obsessed Delivery Playbook?
Yes, absolutely! Check out the colophon for details on options available to your organization.
If you find Customer Obsessed Engineering and the Delivery Playbook valuable, please share it with your friends and coworkers.
Laura Hennifan, Kelly Main, What Is An OKR? Definition & Examples, Forbes Advisor, Jul 20, 2023.