Walk into too many companies mid-pivot to “outcome-driven” and you’ll find the same thing: a roadmap that sounds like a transformation but runs exactly as it did before. Leadership saw the problem, read the books and issued the decree — “no more feature factories.” The team, wanting to comply but not given the time and autonomy to invest in real transformation, reaches for a version of compliance that takes an afternoon instead of a quarter. They go down the roadmap and rename everything.

“Build SSO” becomes “Deliver enterprise authentication.” “Add CSV export” turns into “Enable customer data portability.” “Upgrade the search index” blossoms into “Provide a world-class discovery experience.” Every line is a feature wearing a grander name. The roadmap is the same roadmap. The order is the same order. The way they decide what to build hasn’t budged, and neither has the way they’ll — or won’t — know whether any of it worked.

They’ve run find-and-replace on a feature list and called it a transformation.

This isn’t a strawman, and it isn’t rare — it’s the rule. The move from features to outcomes is the most preached idea in product transformation. It’s also the one most often faked. People adopt the vocabulary in an afternoon and skip the part that actually costs something. Then a year later they’ve shipped everything they promised, the burndown charts look beautiful and somehow nobody — not the customer, not the business, not even the team that shipped it — is any happier.

By happy I don’t mean satisfied in some vague, mood-survey sense. I mean the only thing that actually counts: the customer got a real problem solved, and the business got the change it paid to produce. Shipping features guarantees neither. I want to unpack why that is, and what an outcome actually is, because the gap between renaming your features and genuinely working from real outcomes is the gap between a busy year and a year that mattered.

What an outcome actually is

There’s a line from General Patton that product people love to quote: “Never tell people how to do things. Tell them what to do and they will surprise you with their ingenuity.” Marty Cagan opens his case against roadmaps with it. A conventional feature roadmap does exactly what the General warned against. It dictates the how — build this, then this, then this — instead of naming the what, the result you actually need, and it quietly smuggles in an assumption: that whoever wrote the list already knows which features will produce that result.1

Cagan’s example is a single innocent-looking line: “integrate PayPal.” Why is it there? Is it because a segment of customers genuinely can’t pay any other way? Because international payments are failing? Because the fees are too high? Because a competitor has it and someone got nervous? Each of those is a different problem, and the roadmap line names none of them. Maybe integrating PayPal addresses the real one; maybe it’s beside the point entirely. With only the feature in front of you, there’s no way to judge whether it’s the right bet — and no way to tell afterward whether it worked. The feature is on the roadmap. The reason is nowhere.2

Here’s the distinction the whole topic hinges on. A feature is something you build. An outcome is the difference that thing makes for a customer or for the business. Josh Seiden, whose Outcomes Over Output is the book on this exact subject, compresses it to one line: an outcome is “a change in human behavior that drives business results.” Cagan and Felipe Castro frame the same idea operationally — an outcome is two inseparable parts: a clear problem to solve and a measure that tells you whether you’ve solved it. “Integrate PayPal” is an output. “Reduce checkout abandonment for international customers” is the beginning of an outcome — and the moment you state it that way, you’ll notice that PayPal is now just one of several ideas you might test, not a foregone conclusion.34

That’s the real shift, and it’s why renaming does nothing. “Deliver enterprise authentication” is still an instruction to build a thing. It never names the problem, so it can never be measured, so the team is right back to being judged on whether they shipped — which is precisely the trap everyone claims to be escaping.

How to write one that holds up

The format I use, the one described in Chapter 2.2 Roadmaps and OKRs of the Delivery Playbook, is deliberately boring: we will [objective] as measured by [key result]. The objective is the problem, stated qualitatively. The key result is the measure, stated in numbers. Force both halves to exist and you make it almost impossible to disguise a feature as an outcome, because a feature has no measure of success that isn’t just “we shipped it.”

The most common way this goes wrong is mistaking the work for the result. I’ve seen this version more times than I can count:

Objective: cut warehouse order processing data-quality errors reported to the support desk.

Key result: implement improved CSV to JSON data intake parser to handle empty, missing and null data values reliably.

Implementing fixes to the parser is an activity. It’s the thing you suspect might reduce errors. It is not evidence that errors went down overall. A real key result names the change in the world you’re after:

Objective: cut warehouse order processing data-quality errors reported to the support desk.

Key results:

• support desk data quality issues reduced from over 20 per day to less than 1 per week.

• orders that fail on bad warehouse data reduced from current 7% to under 1%.

Notice those measure customer-visible reality, not internal effort. And notice they’re leading indicators where possible — things you can watch move week over week — rather than a single lagging number you only get to read at the end. Teresa Torres draws this line cleanly: a business outcome like revenue or churn is a lagging indicator, mostly outside any one team’s direct control, while a product outcome is a measurable change in user behavior that the team can move now and that leads the business result. So if the outcome you care about is annual recurring revenue, you don’t wait a year to find out how you did. You pick the behaviors that roll up to it — activation, conversion, the support tickets that signal friction — and move those.5

Castro keeps a running list of what clear problems sound like, and it’s worth internalizing the cadence: help customers solve issues without having to contact us; reduce the time for a user to produce their first monthly report; reduce the subscriber churn rate; connect job seekers with more suitable jobs. Every one names a difference in someone’s experience. None of them names a feature. That’s the test.6

Why renaming isn’t enough

If stating an outcome were the whole job, the move would be easy and far more teams would pull it off. Too often, they don’t, and Cagan is precise about why. Working from outcomes only functions when two other things are already in place.7

The first is context. An outcome handed to a team in isolation isn’t enough — the team also needs the bigger picture of where the company is going, which is the job of a genuine product vision. Without it, a team optimizing its little metric can cheerfully march in the wrong direction. The second is a real product strategy: the reasoning that connects top-level business goals to the specific, prioritized problems each team is asked to solve. Most organizations skip this entirely. They have business goals and they have a pile of feature ideas they hope will hit those goals, and the empty space between the two is exactly where strategy is supposed to live.8

Here’s the typical misstep. A company decides to “do outcomes,” and immediately limits itself to the outcomes its existing dashboards can already measure. Everything else stays a feature. But the most valuable problems are very often the ones you don’t instrument yet — the ones that require defining a new metric and doing the unglamorous work of collecting the data. Letting your current KPIs decide which problems are allowed isn’t rigor. It’s the tail wagging the dog.9

This is also the real reason OKRs earned such a black eye. Thousands of companies layered the OKR ritual on top of an unchanged, output-driven roadmap process and were baffled when nothing improved. The technique didn’t fail them. They’d bolted a product-model practice onto a feature-factory operating model and kept everything else the same — the same funding by project, the same stakeholders dictating features, the same definition of done. You can’t adopt the scoreboard and refuse to change the game.10

This is what I’ve written about in Product teams really do outperform and, more tactically, in Can you rely on SAFe to deliver elite-performer gains?. So renaming alone is nowhere near enough, but an honest shift toward true outcome-based goals is a real beginning. It’s also just half the battle if your ultimate goal is elite-performing results.

Not everything is an outcome — and an honest test

A fair objection: does every line really have to become a problem-to-solve with a metric attached? No, and pretending otherwise is its own kind of theater. There’s always a set of keep-the-lights-on work — bug fixes, a compliance report you now have to file, a tracking pixel a partner needs. Cagan’s advice is refreshingly blunt: unless there’s real risk involved, don’t agonize over the outcome. Put it straight on the backlog and knock it out. Reserve the outcome discipline for the work where it actually changes what you’d build.11

I like to test assumptions. Challenge them by looking for an outcome. A true KTLO emergency — the system is down, get it online fast — is easy to wave off as “exempt” but don’t stop there. Make sure the server doesn’t crash again. Put a metric in place to validate the tracking pixel actually delivers data our partner can act on. “Exempt” means no decision changes based on the answer. Do the work, keep it, and move on no matter what. Measuring it would be theater. That’s true KTLO.

On the other hand, if the result would change what you do next, it’s product work wearing a KTLO costume, and it earns an outcome.

Each passes its litmus test by informing a decision — keep going, change approach or kill it. A measure that informs nothing is a vanity metric.

And if you want to know honestly where you stand today, run the look-back test Cagan recommends. Pull up last year’s roadmap and grade it — not on how many items you delivered, but on how many actually moved the business result they were meant to move. Most teams have never done this, and the first time is uncomfortable, because the delivery rate is usually high and the impact rate is usually grim. That discomfort is the point. It’s the same gap the research keeps finding at the industry level: the majority of shipped features see little or no use. A team measured on output looks productive while quietly building a mountain of work nobody wanted. Outcomes are how you stop mistaking motion for progress.1213

Where to start on Monday

You don’t need a reorganization to begin. You need one honest pass through your current roadmap.

Take it line by line, and for each feature ask two questions: what problem is this supposed to solve, and how would we know if it did? Write the answers down. Some lines will resolve cleanly into a problem and a measure — those were outcomes all along, just mislabeled. Some will expose that you have no idea why the feature is there, which is the most useful thing you can learn. And some are keep-the-lights-on work that belongs on the backlog, no ceremony required. What you’re left with maps almost one-to-one onto a set of OKRs, which is exactly what Cagan means when he says the two are the same idea in different clothes.14

The goal of all this isn’t tidier paperwork. It’s what I call line of sight — the property that any person on the team, asked at any moment why they’re working on the thing in front of them, can name the customer problem it solves and the number that will prove it. When that line of sight exists, you stop shipping features into the void and hoping. You start solving problems and knowing.

You can rename every feature on your roadmap by Friday. It will feel like progress and change nothing. Or you can do the harder, slower work of naming the problems and the measures — and actually have something to show the next time someone asks whether all that shipping made anyone happier. It will count for something — but it’s only base camp on the climb to a true product-team transformation.

For the mechanics of turning objectives into measurable key results, see the Delivery Playbook chapter 2.2 Roadmaps and OKRs, and its companions on 2.1 Product strategy and 1.3 Product vision. For the evidence that outcome-driven product teams genuinely outperform, see Product teams really do outperform: bringing the receipts.

Word-of-mouth is how this newsletter grows, and your support means everything.

Share Customer Obsessed Engineering

In this fourth article on data pipelines we move from architecture to code. Our goal is a working steel thread — a thin slice of the pipeline that touches every major component end-to-end. By the time we’re finished, raw data will flow in from an external source, pass through an anti-corruption layer, land in our internal cache, get transformed and annotated in stages and emerge as something a consumer can read.

We won’t be building a complete application. I’ll assume you can stand up a project, wire up a database and run a web server. What we will focus on is technique — the small architectural moves that make the difference between a pipeline that grows gracefully and one that gets thrown away in six months.

The previous three articles set the stage. We talked about why data pipelines are interesting, the foundational architecture needed to support our requirements and the refinements that make it scalable, resilient and ready to evolve. If you haven’t read them, this article will still make sense — but you’ll get more out of it if you have.

Two (small) sprints, one steel thread

Our architecture has a lot of moving parts: an external data source, an anti-corruption layer (or “ACL,” a term borrowed from Domain-Driven Design that I’ll explain in a moment), an internal cache, a chain of analytical stages, an annotation pattern, an event backbone and a visualization layer. Standing all of that up at once is too much for one sprint. We’ll split the work into two sprints, each producing a runnable system.

In sprint one we deliver the framework — the shape of the pipeline — with every piece wired up except real event streaming. Stages call each other directly through a small in-process API that mimics the eventual event bus. Source data flows through the ACL, lands in the cache, gets annotated through cascading stages and is read out through a simple consumer.

In sprint two, which is the subject of the next article, we evolve that thin pipeline into a true event-driven system. The synchronous dispatcher gets swapped for a message broker. Stages become independent processes. We add the ability to rewind history.

Why split it? By focusing on proving the rest of the architecture first, we’ll tackle a lot of foundational questions. If we tried to do everything at once, the messaging work would dominate our attention and we’d never get a chance to validate the pipeline shape itself — ACL, cache, staged transformations, annotation, readout. Validating that shape first is how we make sure the pipeline actually solves our business problem. We’re holding back messaging — but messaging is a well-understood space. This way, we get rid of more unknowns up front, and later we’ll use the running pipeline to inform what the event contracts should look like.

This is the steel thread approach in miniature. We’re tackling architectural risk early, but we’re also being honest about the order in which we tackle it.

What sprint one actually delivers

Here’s the thing about a steel thread: it’s tempting to over-build. You see the future architecture in your head and you want to get there now. Resist that urge. The point of sprint one isn’t to anticipate every future need — it’s to prove the shape is right.

Concretely, by the end of sprint one we’ll have:

1. An anti-corruption layer that pulls source records from an external SQL database and translates them into our domain types.

2. An internal cache that holds a copy of the cleaned source data plus any annotations we generate.

3. A staged transformation pipeline — a sequence of analytical steps, each one reading the previous data and adding new annotations on top.

4. An annotation pattern that lets us add new derived data alongside the source without touching the source itself.

5. A pipeline-shaped API that looks like an event bus from the outside but is implemented as in-process function calls underneath.

6. A read API that surfaces the latest annotations on demand.

7. A minimal output — for example, a simple web page or even a JSON dump — that proves the round-trip works.

What we’re not doing yet:

• No real message broker. No Kafka, no RabbitMQ, no Phoenix PubSub fan-out across nodes.

• No asynchronous stages. Each stage runs in the same process, in order.

• No rewind-and-reprocess. We’ll design so we can move in that direction, but we won’t have all the pieces in place yet.

• No production infrastructure beyond what we need to run the steel thread end-to-end.

That’s a deliberate choice. Every item on the “not yet” list maps to a piece of work we’ll pick up later. By naming them now, we’re reminding ourselves where the future seams are — and we’re writing today’s code in a way that respects them.

Here’s what sprint one looks like as a flow:

Notice the purple component: that’s the in-process dispatcher — the seam where the eventual event bus will live. Stages publish through it today, and they’ll publish through it tomorrow. Only the implementation underneath changes.

The anti-corruption layer

I introduced the term “anti-corruption layer” in Part 2. It comes from Eric Evans’ Domain-Driven Design and it does exactly what the name suggests: it protects our domain from being corrupted by the shape, types and assumptions of an external system. The ACL is where messy outside data becomes clean inside data.

In our pipeline the ACL has four jobs:

1. Translate external types into domain types. If the source schema stores duration as an Integer of seconds and our domain prefers Time or Duration, the translation happens here.

2. Validate what comes in. Required fields, value ranges, referential integrity — checks at this boundary catch problems where they’re cheapest to fix.

3. Decouple our internal cache schema from the source. If the upstream team adds a column, drops a column or renames a field, the change stops at the ACL.

4. Alert loudly about problems with the inbound data. If the data doesn’t fit our internal shape, we need to know about it.

ACLs are typically shown as small annotations on context boundaries, like this:

We see two internal domains and an external daily calendar domain. Activities and sprints belong to our application. To keep both domains “pure,” an anti-corruption layer translates the external world into familiar, internal language. For example, reading an Integer time duration and translating it into Time. The Activities and Sprints contexts both use the same language, so they don’t need an ACL between them.

I’m using Elixir and the Ash Framework for these examples. Don’t worry if you’re not an Elixir programmer — Ash is a declarative resource framework, and the snippets read more or less like an annotated description of the resource. If you’re working in another stack, the shape of the solution is the same; only the syntax changes.

Here’s a stripped-down resource that represents an Activity in our internal cache. It’s the clean, domain-flavored version of whatever the source database hands us:

defmodule WastePipeline.Cache.Activity do
  use Ash.Resource,
    domain: WastePipeline.Cache,
    data_layer: AshPostgres.DataLayer

  attributes do
    uuid_primary_key :id
    attribute :external_id, :string, allow_nil?: false
    attribute :date, :date, allow_nil?: false
    attribute :duration, :time, allow_nil?: false
    attribute :type, :atom, constraints: [one_of: [:value_add, :non_value_add]]
  end

  relationships do
    belongs_to :stereotype, WastePipeline.Cache.Stereotype, allow_nil?: false
    belongs_to :user, WastePipeline.Cache.User, allow_nil?: false
  end
end

Now here’s the ACL itself — a small module whose only job is to take a source record (whatever the upstream SQL database hands us) and produce a domain record we can hand to the cache:

defmodule WastePipeline.Intake.ActivityAcl do
  alias WastePipeline.Cache

  # Translate one external row into a domain-shaped map. Anything weird about
  # the upstream representation gets dealt with right here.
  def to_domain(%{"duration_seconds" => secs, "value_type" => v} = row) do
    %{
      external_id: row["id"],
      date: Date.from_iso8601!(row["activity_date"]),
      duration: Duration.time_from_seconds(secs),
      type: translate_value_type(v),
      stereotype_id: row["stereotype_id"],
      user_id: row["user_id"]
    }
  end

  defp translate_value_type("VA"),  do: :value_add
  defp translate_value_type("NVA"), do: :non_value_add
  defp translate_value_type(other), do: raise "Unknown value type: #{other}"

  # Public entry point. Pull a batch from the source, translate every row,
  # and hand the result to the cache for insertion.
  def ingest_batch(rows) do
    rows
    |> Enum.map(&to_domain/1)
    |> Enum.each(&Cache.create_activity!/1)
  end
end

Two things worth noting. First, the translation logic is explicit — when the source uses "VA" and "NVA", our translator is the one and only place those strings get mapped into atoms. If the source ever changes (say, the strings become "value-add" and "non-value-add"), exactly one function changes. Second, we fail loudly on unknown inputs. An unrecognized value_type raises rather than silently dropping the row. That’s a discipline worth keeping: the ACL is the place where surprising data gets noticed.

Right now, if something goes wrong the Elixir process crashes loudly. In early development, that’s what we want — to see those exceptions and treat them as the blockers they are. In production, we’ll have a supervisor that shunts the failed message into an error-handling queue and then carries on. We can diagnose, fix and reprocess the message asynchronously.

The ACL is where we earn our data integrity. Once the data is past this layer, every downstream stage gets to trust it. That trust is what lets us write the rest of the pipeline as a series of clean, focused transformations.

DataFrames as the carrier

With clean data in the cache, the next question is: in what shape do we move it through the pipeline? My answer is a DataFrame.

A DataFrame is a two-dimensional table, like a spreadsheet — rows of records, named columns of values. The big advantages are speed and expressiveness. Operations like filtering, grouping, summarizing, joining and adding columns are first-class, and they’re implemented in highly optimized native code.

In Elixir I use Explorer, which is built on the Rust Polars library (if you’re working in Python, you can use pandas or Polars directly). The mental model is the same regardless: a DataFrame is the carrier that moves through your pipeline, and each stage adds to it without altering what came before.

Here’s an example of a DataFrame stage. It takes in activity data, filters out paid time off, groups by user and adds a column for total non-value-add seconds per user:

defmodule WastePipeline.Stages.NvaSummary do
  require Explorer.DataFrame, as: DF

  def run(df) do
    df
    |> DF.filter(stereotype_code != "pto")
    |> DF.group_by(["user_id"])
    |> DF.summarise(
        nva_seconds_sum: sum(nva_seconds),
        va_seconds_sum:  sum(va_seconds),
        total_sum:       sum(total_seconds)
      )
  end
end

Read that pipe-style code top to bottom. We start with df (the DataFrame). We filter out paid time off. We group by user. We summarize into three new columns. The original DataFrame isn’t mutated — DF.filter, DF.group_by and DF.summarise all return new DataFrames. The transformations are pure, the data is immutable and the next stage gets a clean input.

This is what I mean by “DataFrames as the carrier.” Each stage takes a DataFrame in, produces a DataFrame out. Annotations are new data, not modifications. If we get the analysis wrong, we throw out the annotations and rerun the stage; the source data never changes.

A pipeline-shaped API that hides the messaging

Now we earn our keep over the long run.

We want our architecture to eventually be event-driven. Stages publish events when they finish; other stages subscribe and react. But in sprint one we don’t have a message broker yet, and we don’t want to build one. So how do we write today’s code so it doesn’t need to be rewritten tomorrow?

Read more

Your team works hard. The hours are long, the standups are full and everyone looks busy — and yet the thing that should take a day takes three. Velocity feels slower than the effort warrants. Retros surface the same complaints again and again yet nothing moves. If you’ve started to suspect the team is just slow, stop.

It isn’t. The time isn’t evaporating. It’s leaking — escaping through gaps you walk past every day without seeing them. And here’s the part that matters: the gaps aren’t a mystery. There are exactly eight of them, and they were named decades ago, on the factory floors of postwar Japan.

When Dave Brailsford ran British Cycling and then founded Team Sky, he didn’t chase one breakthrough. He hunted one-percent improvements everywhere — the bikes, the riders’ sleep, even the way they washed their hands to dodge a cold before a race. Stack enough of them, the theory went, and a country that had never produced a Tour de France winner takes five of…

Read more

Just about every large pre-Internet company out there today has some sort of “digital transformation initiative,” but what most of them don’t realize is that the heart of this transformation is moving from project-mindset to product-mindset. — Marty Cagan, Founder, Partner, Product at SVPG

Forty-plus people, a chunk of a week blocked off, a cavernous meeting room commandeered. Sticky notes color-coded by team. Butcher paper taped along the walls. Somewhere near the front, a Release Train Engineer is walking the room with a Sharpie, drawing red strings between dependent features. By the end of day two, every team has its slice of a 10-week plan committed to the wall, and a senior leader is asking each team for a five-finger confidence vote.

I’ve watched that scene play out. I know what comes next. Six weeks from now, three of those red strings will have moved, two features will be quietly cut and one team will be rebuilding a thing nobody asked for. The plan that got committed is not the plan that ships. Everyone in the room knows. Nobody says it.

But the plan got a vote. The Release Train left the station. It’s now truth — until the next PI, when a slightly different plan will become the next truth. The change orders get signed and project budgets run over in the guise of “increased scope.”

That’s PI Planning. And I’m going to argue, carefully and with receipts, that what just happened in that room is waterfall on short cycle — and almost nothing about it is recognizably agile.

The premise we're not going to relitigate

I am not going to relitigate whether product teams outperform. I made the case in Product teams really do outperform: bringing the receipts. DORA, McKinsey, Pendo, Standish and Project Aristotle converge unambiguously. Small, durable, cross-functional teams that own a problem end-to-end deploy 182× more frequently, ship higher quality, redefine “done” around outcomes and drive four to five times faster revenue growth than the alternatives.1

Take that as given. The question for today is narrower: if that is what real agility looks like, where exactly does SAFe land on the map?

The short answer — the one I will spend the rest of this article defending — is that SAFe lands far closer to waterfall than its name implies. Not because its proponents are dishonest. Because its structural unit of planning, funding and authority is the Agile Release Train, not the team. And that single design choice quietly recreates every handoff, gate and silo agile was meant to eliminate.

Before I break the case, let me make it

I owe SAFe and the people who chose it a fair hearing — and a fair hearing means arguing with its strongest version, not its sales deck. So let me make SAFe's case the way its creator makes it.

Dean Leffingwell does not justify SAFe by appeal to certifications or board-deck optics. He grounds it in flow economics — specifically Don Reinertsen's Principles of Product Development Flow, a serious book that the continuous-delivery world draws on too. The argument lives in SAFe's seventh principle, “apply cadence, synchronize with cross-domain planning,” and it runs like this. “Solution development is an inherently uncertain process… This risk conflicts with the need for businesses to manage investments, track progress, and have enough confidence in future outcomes to plan and commit to a reasonable course of action.” The resolution Leffingwell offers is a, “safety zone, where enough uncertainty actually provides the freedom to pursue innovation, while sufficient confidence allows the business to operate.” Cadence and synchronization are how you buy that confidence without freezing the work. PI Planning, in this telling, is not a commitment ritual — it is a periodic re-baselining of the plan against the objective state of working systems, which is SAFe's fifth principle.2

Take that seriously. The coordination problem SAFe addresses is real. Eight teams shipping into the same product surface need to know about each other's plans. Two hundred teams across a portfolio need someone to decide the trade-offs. Lean Budgets — funding value streams instead of approving each project individually — are genuinely better than the alternative most enterprises were running before. Inspect & Adapt at the end of each PI is a retrospective at scale, and that’s not nothing. PI Planning, for all its flaws, beats no planning. At the team level, SAFe leaves Scrum or Kanban in place. The bottom-up practices are still there.

There is also a colder, political version of the case, and it deserves the same hearing. SAFe is, in many enterprises, the only model executive leadership will fund. The CFO can read it. The PMO can map their roles into it (that’s a key point; hold on to it). The risk officer can find their gates. There is a certification track, a vendor, a roadmap. For an organization that needs to tell a board, “we are being agile,” SAFe is the politically survivable option. The choice between SAFe and a Cagan-style product transformation is rarely the choice on the table. The actual choice — for many large organizations — is between SAFe and the project-funded, requirements-thrown-over-the-wall, six-month-release waterfall it replaced.

So the honest steelman has two layers. Intellectually, SAFe is an attempt to operationalize real flow economics at enterprise scale. Politically, it is the floor, not the ceiling — and the floor is higher than what came before. For organizations starting from heavy-handed waterfall, SAFe is an upgrade on both counts.

That is the strongest case I can make, and I built it from SAFe's own foundations. Now let me explain why it isn't enough — and why the upgrade comes at a steeper price than its buyers usually realize.

Five tells that betray the waterfall underneath

If you want to know whether something is waterfall, do not read its marketing. Read its structure. Waterfall has diagnostic attributes — sequential phases, big-batch commitments, multi-team handoffs, stage gates with formal approval, plan-driven coordination above team level — and any process that exhibits those attributes will behave like waterfall whether or not it carries the name.

What follows are five attributes that SAFe exhibits, with the practices that produce each. Some of these will look familiar from the structural critique Marty Cagan put on the record in 2018. Others are mine, drawn from watching the framework operate across enterprise programs. The attributes are independent. Any one of them is a yellow flag. Three of them is a diagnosis. Five of them is what we are looking at.

Tell #1: the Release Train is a waterfall, on rails

Start with the name. Agile Release Train. The naming is not accidental — it is defensive. Strip the adjective and you are left with release train, which is exactly what waterfall organizations have always called their integrated, scheduled, multi-team delivery cycles. Pull a release train out of a 1995 PMO and it would have looked nearly identical to a SAFe ART: 50 to 125 people working toward a synchronized release at a fixed cadence, a release manager coordinating dependencies, a portfolio manager approving the work and a steering committee voting on the plan. SAFe replaced the release manager with a Release Train Engineer, the portfolio manager with a Lean Portfolio Manager and the steering committee with a PI Planning ceremony. The roles got new titles. The meeting got a Sharpie. The cadence shrank from six months to ten weeks. The shape did not change.

Look at what an ART actually does. It locks 50 to 125 people into a shared 10-week commitment. It synchronizes their backlogs into a single Program Backlog. It coordinates their dependencies — through a Solution Train, when the ART is too small to contain the dependencies on its own. It runs ceremonies — PI Planning, ART Sync, Scrum of Scrums, System Demo, Inspect & Adapt — whose entire purpose is to manage the friction created by the ART's own existence. Every one of those ceremonies is overhead a real product team does not pay, because a real product team does not have a 50-to-125-person commitment to coordinate around in the first place.3

Marty Cagan put this plainly enough that I want to quote him directly. In Revenge of the PMO, he writes that, “the critical notion of a dedicated product team (aka squad) has been gutted. In SAFe, this concept of a true product team has been undermined and demoted, and the core concept is now a Program, which has a top-down model of a product manager, an architect, and a release train manager, and these people make all the key decisions, and then some number of engineering teams with a low-level product owner are assigned various parts to build.” That is a verbatim description of a feature team inside a project structure. SAFe just renamed the project a Program and the project manager an RTE.4

Dave Farley — co-author of Continuous Delivery and not a man with a Scrum axe to grind — reached the same place from the engineering side. Writing about the clients he had watched adopt SAFe, he was blunt: “the orgs that I have seen try SAFe look indistinguishable from the orgs that pay lip-service to Scrum to me. In both cases, they look like waterfall orgs to me.” His objection is structural, and it lands on the load-bearing artifact directly. He dislikes the Release Train itself — “I think that they are an anti-Continuous Integration step,” one that, “represent[s] a plateau that halts team's progress towards a more effective flow-based approach.” The author of the canonical book on continuous delivery looked at the Release Train and saw the thing standing between teams and flow.5

Pull the Release Train out and SAFe collapses. Keep it in and you have kept every structural property that makes waterfall what it is — synchronized big-batch commitments, multi-team handoffs, a coordinator above the team, a plan that hardens the moment the room votes on it.

Tell #2: cadence as commitment

A 10-week PI plan is functionally a specification.

SAFe will tell you it is a forecast — soft alignment, not a frozen commitment. Confidence votes are five-finger, not yes-or-no. Teams can replan. Deployment is not locked to the PI. The wording is all soft. Look at what actually happens, though. The room agrees on a set of features for the PI. Those features get pulled into the Program Backlog. Engineers start work against them. Dependencies get committed to other teams' plans. Stakeholders are told what is coming. Mid-PI, the cost of dropping a feature is not zero — it is the team's confidence vote, the dependent teams' rework, the stakeholder's roadmap update, the RTE's coordination tax, the System Demo's missing slot.

So features almost never get dropped mid-PI — the discovery that should happen during the increment rarely does, and when it surfaces a problem, the dependencies make dropping the feature more expensive than shipping it anyway. The plan, framed as soft, hardens into something tighter than waterfall ever was — because waterfall at least admitted to being a commitment. PI Planning insists it is not a commitment, then behaves like one, and punishes anyone who treats it as if it isn't.

Compare this to a real product team. A team starting Monday morning with a hypothesis runs the cheapest possible test by Friday. If the data says no, the hypothesis dies before week two. The team has lost five days. No roadmap vote, no red strings. The stakeholder isn’t told anything because there is nothing to tell. One of these is research. The other is a Gantt chart that happens to be 10 weeks wide instead of 10 months wide.

Royce warned about this in 1970 — and it is worth remembering, because the diagram he drew has been the most consequential misreading in software engineering history. In Managing the Development of Large Software Systems, Royce presented the now-familiar sequential flow of phases and then said in the same paper that an implementation plan keyed only to those steps was “risky and invites failure.” The bulk of his paper argues that any large software system needs to iterate — that every commitment to a plan should be tested against working software before it hardens. People extracted his diagrams and ignored his warnings. PI Planning ships Royce's exact failure mode under a different name, on a shorter cycle. The cadence being shorter is the entire thing that is “agile” about it. Strip the cadence and what is left is a stage-gated portfolio process.6

Tell #3: the decomposition pyramid

Epic → Feature → Story is requirements → design → implementation, in different fonts.

The Epic is approved upstream by Lean Portfolio Management before it lands at the ART. The Feature is decomposed by a Product Manager and a System Architect before it lands at the team. The team's Product Owner — a role title borrowed from Scrum and stripped of Scrum's authority — turns Features into Stories. Each step down the pyramid is a translation from one set of vocabulary to another. Each translation loses information. By the time a Story is in front of a developer, the original customer intent has been refracted through three layers of people who did not sit with the customer and who do not sit on the team that will build the thing.

Where, exactly, does product discovery happen?

Discovery — as Cagan, Torres and the discovery-coaching tradition mean it — is hypothesis-testing with real customers before commitment. In SAFe, that work happens before the Epic enters the funnel, in a separate process, performed by people who do not sit on the team that will build it. The feature team receives a Feature already shaped by a Product Manager, refined by a System Architect, approved by Lean Portfolio Management and committed to by 50 to 125 of their colleagues at PI Planning. The team's role is to decompose the Feature into Stories and execute. That is not discovery. That is a business analyst with new vocabulary.7

I am not the first person to point this out. Jeff Gothelf, the co-creator of Lean UX, put it bluntly:

“All the principles we've built into Lean UX don't seem to exist in SAFe.”

He went on to note that customer-centricity, evidence-based decision-making, experimentation and course correction, “are visibly absent from the SAFe conversation.” He is right because the conversation cannot include them. The framework has placed those activities upstream of the team, and made the team's slot in the value chain a unit of execution, not a unit of inquiry.8

The Delivery Playbook focuses on customer value as a core practice — through value stream engineering and customer-centric design. These are not optional — they are table stakes. SAFe cuts both out of the conversation.

Tell #4: the team-ownership ceiling

A SAFe team does not own a value stream. It owns its slice of an ART.

End-to-end ownership of the customer outcome — what separates product teams from feature teams — lives at the ART or Solution Train level. That is the line Receipts drew with Cagan's framing of product, feature and project teams. SAFe puts the team on the wrong side of that line by design. The framework cannot fix this because the line is the design. The Release Train is the unit of ownership. The team is the unit of execution. Move the unit of ownership down to the team and you do not have an ART anymore — you have a portfolio of small product teams, which is what Cagan, Team Topologies and DORA's elite-tier qualitative findings all converge on.9

This is the structural reason SAFe transformations stall. The ceremonies improve. The ART runs more smoothly. PI Planning gets more efficient. Nobody on the team feels more agile, because nothing about the operating model above the team has changed. The team is still being handed Features it did not discover, against a 10-week commitment it did not negotiate, with deployment dependencies it cannot break. The agile-coach stack is workable. The teams are not.

Teams cannot build ownership in what they don’t own.

Tell #5: Lean Portfolio Management is the PMO with new business cards

The title is mine; the diagnosis is Cagan's — his article is literally called Revenge of the PMO.10

Look at what Lean Portfolio Management actually does. It captures Epics in a Funnel. It moves them through Reviewing and Analyzing states. It requires a Lean Business Case for each Epic before that Epic can advance. It gates approval at a Portfolio Backlog state. It tracks Implementation against the original case.

State the same workflow in 2005 vocabulary: capture projects in an intake queue, move them through analysis with required business cases, gate approval at portfolio review, track implementation against the original case. The verbs are identical. The artifacts are identical. The decision rights are identical.

SAFe's defenders will say Lean Budgets are different — funded to value streams, not projects — and that is fair, in principle. Funding value streams is a real improvement over funding projects. But the governance overlay — the gates, the cases, the approvals, the portfolio reporting — is the project portfolio management discipline that PMOs have run for thirty years. It’s not bad because it’s old. It’s bad because it sits above the team and makes the team's authority subordinate to the portfolio's case-building exercise. Authority for “what to build” still flows top-down. The team gets to decide how. That is exactly the bargain SAFe strikes: the project mindset wearing a costume.

If you have a PMO that has been quietly converting itself into Lean Portfolio Management, the costume is the only thing that has changed. The decisions are still being made by the same people, in the same room, against the same business cases, with the same approval ladder. Agile pushed the PMO to the side; Lean Portfolio Management is its path back into the room.

The DORA test

Here is the test I keep coming back to. It is short. It is fatal.

Read more

Marty Cagan’s decade-old article can be a real rabbit hole. It’s genuinely hard to stop reading his blog, especially when digging up old gems like this.

Here’s something that resonates right now: “Good teams are skilled in the many techniques to rapidly try out product ideas to determine which ones are truly worth building. Bad teams hold meetings to generate prioritized roadmaps.”

Most teams I meet are funded, staffed and measured entirely around output. Hit the date, close the ticket, ring the bell for delivering on a roadmap.

Too few are paying attention to what matters: did what we shipped really make a difference for our customer?

The reorg that actually changes delivery isn’t new ceremonies. It’s moving the moment you celebrate from “we shipped” to “it delighted our customer, it actually moved what matters.”

Read more

Your team has probably spent more hours this year waiting on pull requests than it spent writing the code marooned inside them. Andrea Laforgia opens his March 2026 essay Stop Using Pull Requests with numbers from across the industry that count the cost. The piece is the most thorough reckoning I’ve seen of why the modern pull request workflow is mis-designed for the teams that use it. Go read it. Then come back here — because if you’re using the Delivery Playbook, Laforgia is describing the practice you already know. He just gives it a name. And I like it.1

That name is T*D — Test-Driven Development, Trunk-Based Development and Team-Focused Development, fused into one operating model. Hold on to it. We’ll come back to it.

Don’t break the flow

Chapter 2.8 Delivery processes & tools lays out what a well-wrought delivery pipeline does: ephemeral environments, test batteries, zero trust, release orchestration, observability, canary releases. The argument is t…

Read more

Six people. One sticky.

A SWOT analysis at the top of a planning workshop. Strengths quadrant — a dozen sticky notes inside ten minutes. Opportunities, fine. Threats, fine. Weaknesses — one sticky.

The team has been together a year. Mixed group, some senior, some junior, some remote and some local. They’re smart — and obviously not weak in just one way. But one is all they were willing to write down, in front of each other, with their names attached. So the Weaknesses quadrant sat there, embarrassed and underfed, while the rest of the wall filled out around it.

Most retrospectives end this way. Most planning meetings, the same. The team knows. The team won’t say.

There’s a 30-minute exercise that breaks this open. It’s not subtle. It’s not high-tech. It’s a borrowed word from Soviet engineering, an inversion trick from a 19th-century mathematician. It takes a team from laughter to honesty to commitment in less time than your average standup. It’s called TRIZ — the Liberating Structures version, which I’ll come back to — and it’s the cheapest unlock I know for a team that won’t speak its own dysfunctions out loud.

Here’s how it works, why it works and how to run it next week.

The wrong question

The silence in that Weaknesses quadrant is not random. It’s the predictable shape teams take when the question goes the wrong way.

“What are we doing wrong?” is one of the worst questions you can ask. It demands that someone you work with every day be the source of bad news, of blame — in a setting where their answer goes on the record. The forces working against honesty are stacked: the social cost of singling out a behavior, the political cost of singling out a person, the personal cost of being the one who said the thing nobody else was willing to say. So people hedge. They write harmless stickies. They give safe answers and stay quiet about the rest.

This isn’t about psychological safety being too low, exactly. It’s about requiring more safety than most teams I’ve known. An unusually safe team — the rare ones — can handle the direct question. Most teams can’t, even good ones. The Goldilocks zone for direct critique is narrow, and most teams sit just below it.

The wrinkle: the same team, asked the question a different way, will answer it. Same people, same room, same dysfunctions. The information is there. The mechanism for getting it out is what’s broken.

That’s the gap TRIZ fills.

Inversion

The 19th-century Prussian mathematician Carl Jacobi had a habit, when stuck on a problem, of saying man muss immer umkehren — “one must always invert.” If you can’t see how to make a thing succeed, see if you can describe how to make it fail. Charlie Munger, Warren Buffett’s late partner, popularized the move a century and a half later: “Invert, always invert,” he’d repeat, crediting Jacobi each time.1

Inversion is a cognitive shortcut. The forward question — “how do we make this succeed?” — runs into the usual social and intellectual walls. People hedge. People defer to whoever spoke first. People say what they think they’re supposed to say. The inverse question — “how do we guarantee it fails?” — jumps every one of those walls, because there’s no commitment. They’re describing a failure. They can be specific, vivid and sometimes uncomfortably accurate, and the room laughs instead of getting defensive.

Once the failure is on the wall, naming the parts of it that already exist is a much smaller step. You’re not confessing. You’re confirming.

That’s the whole engine of TRIZ.

Why it works on teams that won’t speak up

If you’ve never run TRIZ you’re probably thinking, “why does this work at all?” Why does asking a team to describe failure produce honesty when asking them to describe weakness doesn’t? It’s the same people in the same room talking about the same problem.

Here’s four reasons I keep coming back to. They aren’t independent — pull on one and the others move — and there are probably more I haven’t named. Their common thread: all four lower the cost of saying the thing.

Hypotheticals are cheap. Saying, “we could skip estimation entirely” carries no social cost. Nobody is suggesting it. Saying, “we do skip estimation” costs you — especially if the person who set up the no-estimation pattern is in the room. TRIZ collects the cheap version first. It builds a list of behaviors the team is willing to discuss because the team isn’t yet committing to whether they’re real.

Confirmation is easier than confession. By the time Movement 2 starts, the behavior is already on the wall. The team isn’t introducing a new criticism — that’s the hardest move. It’s confirming an existing one — that’s a much smaller move. “Ah, yea, we do that,” is far easier to say than, “we have a problem with X.” The board already said it. The team is just agreeing.

Laughter changes the room. Ten minutes of laughing at exaggerated worst-case behaviors changes the social register. The same conversation about the same behaviors, started cold, would feel like an accusation round. Started after the comedy, it feels like camaraderie. Defensiveness drops. People interrupt themselves. The room is in a different mode.

The quiet people speak. This is the move I see most reliably. In nearly every TRIZ I’ve facilitated, the first person to circle a behavior in Movement 2 — the first to commit to, “yea, that one” — has been the person who hadn’t said much yet. The senior engineer who watches more than she talks. The new hire who’d been listening for three months. They’d seen the dysfunction the whole time. They didn’t have a frame to say it. TRIZ gave them one.

You can put psychological safety theory on this if you want. You can talk about face-saving, about politeness norms, about the asymmetry between criticism and praise. All of it holds. But the practical version is simpler: it’s a lot easier to say things about hypotheticals than about reality, and TRIZ exploits that gap to get a team to say what it already knows.

That’s why a 30-minute exercise can do work that an hours-long retro can’t.

The three movements

The exercise has three movements. Together, they take 30 minutes if you keep the timing tight; you can stretch to 45 on a hard problem or compress to 20 with a small confident team. Each movement has a single question, and each question is doing different psychological work.

Movement 1: the worst-possible-outcome brainstorm (10 minutes). Frame the outcome the team cares about — high-functioning sprint planning, a retrospective that actually changes things, a release process you’d be willing to demo. Then ask the inverted question: what could we do to guarantee the worst possible version of this? Wild, exaggerated, funny answers welcome. Suppress nothing. Two minutes of silent sticky-writing first, in 1-2-4-All style, then a round-robin share.

The laughter is the point. A team that’s laughing at “we could just cancel the retro and send an email instead” is a team becoming willing to say real things. Don’t rush this part — the comedy is what unlocks the next movement.

Read more

I was chatting with a friend about team dynamics. We’d been talking about restructuring around product teams — the kind of empowered, durable, cross-functional teams I’ve written about before. He’s nodding along, asking sharp questions. Halfway through, he pushed back.

I’m paraphrasing, but the gist was, “Look, I get the theory. But where’s your proof? We’ve all heard the pitch. Cross-functional teams, durable squads, outcome metrics, two-pizza this, empowered that. Show me the data. Show me that any of this actually beats what we already do.”

It was a fair question. A great question, actually. I’d written the case in Building a product oriented team and the prequel, Moving away from the Monolith — faster, better with a product mindset. But those pieces argued from reasoning and experience. They didn’t bring the math.

So today I want to bring it. Because the math is there — it’s been there for over a decade — and once you line it up, the conclusion is hard to escape: product teams, the real ones, ship faster, build less waste, deliver higher quality and produce dramatically better business outcomes than the alternatives. The evidence isn’t a single bombshell study. It’s a stack of independent research, all converging on the same answer.

Let me lay out the receipts.

One note before we start. None of the studies I cite below were designed to answer the question “do product teams outperform?” They were designed to answer narrower, related questions — about deployment, about feature usage, about team effectiveness, about developer velocity. What’s below is my synthesis: the act of lining up findings from different research traditions and showing where they converge. The convergence is the argument. Each citation is a thread; the rope is the pattern across them.

A quick refresher on “product team”

Before the data, let’s get precise. “Product team” gets thrown around so loosely it’s often meaningless.

Marty Cagan and the Silicon Valley Product Group draw the cleanest distinction. A product team is a small, durable, cross-functional unit — typically a product manager, a designer and a handful of engineers — that owns a problem and is held accountable for outcomes. They are given problems to solve, not features to ship. The product manager owns value and viability. The designer owns usability. The tech lead owns feasibility. Together they own the result.1

A feature team looks similar on paper — same disciplines around the table — but is handed a prioritized list of features to build. Outcomes belong to a stakeholder upstream. The team is judged on output: did you ship what was on the roadmap? Velocity, story points, throughput.

A project team is even further from product orientation. It’s pulled together to execute a discrete project, then dissolved. Specialists rotate in and out. There’s no durable home for product knowledge.

The labels matter less than the operating model. As I wrote in Building a product oriented team, the real distinction is durability, ownership and empowerment — long-lived teams that own the whole product end-to-end and have the authority to figure out the how.

Now — what the research says.

Receipt #1: they ship dramatically faster

Start with DORA — the DevOps Research and Assessment program, founded by Dr. Nicole Forsgren and now part of Google. Every year since 2014, DORA has surveyed practitioners — more than 3,000 in 2024 alone, and tens of thousands across the decade-long program — and statistically grouped them into performance tiers based on four metrics: deployment frequency, lead time for changes, change failure rate and mean time to restore service. The methodology, the math and the conclusions are documented in Accelerate — the book that emerged from the research and won the Shingo Publication Award.2

The 2024 Accelerate State of DevOps report is the latest. Elite-tier teams deploy 182 times more frequently than low performers, with 127× faster lead time for changes. Elite teams hit lead time under a day, deploy on demand and are roughly 19% of the surveyed population. Low performers measure lead time in months.3

That’s not a marginal advantage. It’s not “20% better.” It’s two orders of magnitude.

You might be tempted to dismiss this as a DevOps story rather than a team-structure story. But every time DORA digs into what differentiates elite teams, the answers are the same attributes that define product teams: small, cross-functional, owning the whole pipeline from code to production, empowered to make their own technical decisions. The opposite — handoffs across functional silos with stage-gate approvals from external groups — correlates strongly with low performance. Accelerate presents one specific finding bluntly: external change advisory boards do not improve stability. They make lead time worse without making anything safer.4

McKinsey’s “Beyond Agile” work, which I cited in How waterfall is wrong for software, puts a number on the same effect from a different angle. Across their consulting engagements, McKinsey observed that companies could compress the average time from “code complete” to live production from 89 days to 15 days when they reorganised around DevOps and product-team principles. Same basic architectures, same kind of features. Six times faster, just from changing how the team takes work from “done” to “deployed.”5

The simplest summary: product teams own the value stream end-to-end. Feature and project teams hand off across silos. Handoffs cost time. Time turns into distance from the customer. Distance from the customer means everything you ship is stale.

Receipt #2: they get more done — but redefine what “done” means

“More done” is the wrong metric. The right metric is: did anyone use what we shipped?

In 2002, Jim Johnson — chairman of the Standish Group — presented a finding at the XP conference in Sardinia that’s been quoted ever since: across enterprise applications, only 7% of features are “always” used, 13% are “often” used and 16% are used “occasionally.” That leaves 64% of features as either rarely or never used. Standish later extended the analysis to claim that roughly 80% of features deliver low or no value. On large projects, Standish data shows organizations deliver only 42% of the features they originally scoped. The Standish numbers have drawn methodology critiques over the years — fair enough — but more recent work points the same direction. Pendo’s 2019 Feature Adoption Report, drawing on telemetry from hundreds of B2B SaaS products, found roughly 80% of features are rarely or never used.6

Sit with that for a moment.

Now, “rarely used” isn’t synonymous with “no value.” Some rare features are quietly essential — compliance flags, account recovery, the audit trail your CISO will ask about exactly once. Both Pendo and Standish gloss over that distinction. Even granting it generously, though, the ratios still point at a meaningful misallocation: a lot of effort going into features customers rarely or never engage with. (I’m looking at you, Microsoft Office).

A feature team measured on roadmap output is, by these numbers, building a sizable wedge of low-value work. They look terrific on burndown charts and terrible on customer impact.

I’ve watched this play out enough times to be cynical about it. A feature team gets a quarterly roadmap. They estimate the work in story points, deliver on schedule, demo a sleek end-to-end flow and get applauded. Six months later, telemetry shows the feature gets touched by 3% of users. By then the team has moved on to the next item on the roadmap. Nobody is accountable for the gap between “shipped” and “mattered,” because shipped is the goal.

A product team would have caught that earlier. They wouldn’t have built the full feature on a quarterly roadmap; they’d have built the cheapest possible version that tested the hypothesis — a prototype, an A/B variant, a deliberately ugly first cut — and watched the data. If the data said 3%, they’d have killed the feature in week two and gone after a more promising hypothesis. That’s the discovery discipline Marty Cagan and Teresa Torres write about extensively, and it’s the operational reason product teams build less and deliver more. Not because they’re slower. Because they aim differently. They invest in discovery before commitment. They kill a feature that doesn’t earn its keep. They redefine “done” from “shipped” to “moved the metric.” And the metric they move is customer behavior, not story points. (I’ve made the same argument from a measurement angle in Stop measuring effort.)

There’s a parallel finding in McKinsey’s What makes product teams effective? — a five-year study of more than 1,700 teams across 75 organizations. McKinsey identified two structural conditions that consistently predict throughput and velocity improvements: dedicated team membership (no context switching across projects) and team persistence of three to six months minimum. Teams meeting both conditions develop reliable estimates and sustained throughput. Teams that don’t, can’t.7

That finding isn’t surprising if you think about how knowledge accumulates. Product orientation depends on knowledge durability — context that only forms when the same people work on the same problem long enough to internalise it (more on this below). Project teams dissolve before that happens. Feature teams reorganise around the next quarterly priority. Product teams stay put. That’s why their throughput compounds.

Read more

A junior developer told me they’re using Claude as a mentor. They liked the patience that never wore thin, the availability that never closed, the way they’re never made to feel doltish.

Here’s what worried me: they said mentor. Not tutor. Mentor.

I don’t think they’re wrong about what they’re getting. But mentorship isn’t about answers. It’s about being seen — watched while you struggle, nudged when you’re avoiding hard conversations, sent on projects that stretch you in ways you didn’t ask for. Mentors have the nerve to tell you the truth in private because they’re invested in who you’re becoming.

AI can’t do that. Because it isn’t invested in you. Because it’s not “in the room.”

What AI is genuinely good at

Let’s start with what’s actually working, because this isn’t a piece about being suspicious of the tools. The tutoring story is real. It’s the part of the AI hype that’s actually earned.

Benjamin Bloom’s famous 1984 paper described what he called the “2 sigma problem:” students who received one-on-one tutoring performed two standard deviations better than students in conventional classrooms — that’s the difference between an average student and the top 2 percent. The catch was scale. Personal tutors are expensive, and most students will never have one. For forty years that gap has been the unsolved promise of education technology.1

A well-configured AI is one way we’re closing it. When Khan Academy rolled out Khanmigo, built on GPT-4, the pitch was exactly this — a patient, infinitely available one-on-one tutor for every learner. In specific narrow tasks, the early returns are hard to dismiss. Harvard published a physics study in 2024 that found students using a properly designed AI tutor learned over twice as much in less time than their peers. They reported feeling more engaged and motivated.23

I see the same thing in software. With the right model, a serious system prompt and a few well-built skills, AI can:

• Walk a junior through a debugger session, naming what each frame means and why it matters.

• Explain why a piece of code is the way it is — not just what it does — and adjust the explanation when the question lands a level too high.

• Run unfamiliar code, watch it fail and reason about the failure with the learner in real time.

• Drill someone on a foreign language, a SQL idiom or a regex until it’s automatic.

• Take the same question for the third time without sighing.

That last one matters more than people admit. A great deal of human learning gets blocked by the social cost of asking again. Removing that cost is genuinely useful. It’s what the junior in my opening responded to. They felt safe.

How tutoring goes wrong

It’s useful, but not magical. The same AI cheerfully invents APIs that don’t exist, pattern-match on the wrong context and confidently produce code that looks right but isn’t. Apple researchers showed last year that current LLMs aren’t doing genuine reasoning at all — performance on math problems dropped by as much as 65 percent when the team added a single irrelevant clause to the prompt. Andrew Zuo’s piece on the “-10X developer” makes the harder point: when an LLM writes code at or above your level and introduces a subtle bug, you may be the worst possible person to find it.45

A newer concern is harder to wave off. An MIT Media Lab study published in mid-2025 used EEG to compare brain activity across people writing essays with no help, with a search engine and with an LLM. The LLM group showed the weakest neural connectivity and the worst ability to recall their own writing minutes after producing it. The researchers called it “cognitive debt.” That tracks. Outsource the struggle and the learning never lands.6

So a serviceable tutor, yes. A replacement for thinking, no. None of that, though, is the central point. The central point is that tutoring and mentoring are not the same job, and the public conversation keeps blurring them.

What a mentor actually does

If you ask me what my mentors did for me over the last forty years, almost none of it was answering technical questions. The technical content I could get from books, from peers, eventually from Stack Overflow and now from Claude (as long as I’m careful and look for fantasy hallucinations). What my mentors did was something else entirely.

Read more

In 3.6 Specification we worked through the first two stages of Behavior Driven Development — Discovery and Formulation. We surfaced scenarios in three-amigos conversations and shaped them into Gherkin “Given-When-Then” statements. That gave us the outline of every feature in our product increment.

Elaboration is where we move from outline to depth.

A very important takeaway from chapter 3.6 bears repeating here, because the entire purpose of this activity hangs on it:

The specifications you write now are living documents. They will evolve, change and be added to. That’s why it’s so important to create a live link to the implementation — the code that will validate the specifications. As the specifications change that live link will preserve our value chain from customer desires to delivered code.

Hold on to that idea. Elaboration isn’t about freezing requirements before the team writes a single line of code. It’s about building the connections — between the spec and the marble, between the marble and the test, between the test and the production code — that let the spec keep evolving without the value chain ever going slack.

Introduction

This activity has three jobs.

First, we expand each scenario across the levels at which the system is tested: component and assembly. A login feature has UX scenarios about what the customer sees and experiences. It also has assembly scenarios about which events fire, which commands route through which aggregates and what payload comes back. Both layers need specifications. Both layers need to be testable. Skipping a layer means leaving a gap through which defects love to slip.

Second, we create a live link between every specification and the engineering model that implements it. Each Gherkin feature points back to a marble. Each marble points forward to a Gherkin feature. That bi-directional link is what makes our specifications living documents — when the spec changes, the marble updates with it, and vice versa.1

Read more

Picture this: a program manager wants to confirm a design decision before a release. She asks, innocently enough, “Can someone point me at the current version of the architecture document?”

A pause. Then: “I think it’s in the Teams channel — the one from the February workshop.” Someone else chimes in, “No, there’s a newer one. Sriram emailed it last week.” A third voice: “That’s the draft. The signed-off version is in the SharePoint site. But the link in the meeting invite is broken because they restructured the folders.” A fourth: “I have a copy in OneDrive. It might be the latest, I’m not sure.”

Five people. Five different versions. Zero confidence that any of them is authoritative. And this is a team that cares about documentation.

The common denominator is always the same: a collaboration stack built on SharePoint, Teams and Microsoft’s various Office products. Sometimes with Azure DevOps (ADO) bolted on to handle tickets and code.

Here’s an argument that will sound harsh to anyone who’s invested in Microsoft 365 as their delivery platform: the Microsoft collaboration stack is diabolically deficient for serious product delivery. It wasn’t built for the problem delivery teams actually face. It has, piece by piece, gone after what enterprise IT wanted to solve in 2007: replacing the file share.

That’s not the same problem.

What a delivery team actually needs

There are five table stakes a team needs. I describe the first three in Getting started: Using the Delivery Playbook. I’ll restate those briefly here, along with the other two:

1. Live collaboration. Multiple people must be able to work on the same artifact at the same time, editing simultaneously, seeing each other’s cursors, throwing virtual sticky notes on a board in real time. Design work is a collaborative act. The moment one person has to collect and collate feedback, fidelity drops off a cliff.

2. Version control. Every change, by every author, is always captured and reversible. Not just “saved revisions” — the full, granular history, searchable, with diffs, always available. This is table stakes. Git has had it for two decades. Any tool that tracks less than Git doesn’t take the goal seriously.

3. Immutable linking. A document’s URL must never change. Not when the file is renamed. Not when it’s moved to another folder. Not when the folder is reorganized. Not when the author leaves the company. A reference made today should still resolve in five years. Without this, every link in every other document becomes a ticking time bomb.

4. Workflow integration. The documentation platform has to talk, fluently and deeply, to the work-tracking platform. Specs have to link to tickets. Tickets have to link to pages. Status changes in one system should not only be visible in the other, but trigger cross-tool workflow changes and render live embeds, not stale URLs. When that integration is tight, your specs, documentation and tickets stop drifting apart — they become a single living view of the work, not several systems the team has to reconcile.

5. Universal, web-first access. One consistent experience for everyone, on any device. Not a degraded web version and a feature-rich desktop app — a single canonical surface, served through the browser, that everybody gets equally no matter what hardware they’re on.

These five criteria are the measuring stick. Let’s hold SharePoint — and Teams and ADO — up against them.

Links that rot

This is the most damning, and the one I want to spend the most time on, because it’s the one almost nobody takes seriously until they’re knee-deep in the damage.

In SharePoint, a document’s URL is a function of its name and its location in the folder hierarchy. Rename the file, and the URL changes. Move it to another folder, and the URL changes. Move the whole folder, and the URLs of every file inside it change. You can turn on “managed metadata” and “content types” and a bunch of other gears that promise to fix this, but in practice — across every tenant I’ve ever touched — they don’t. The URLs change. References break.

Microsoft has shipped “durable links” and “document IDs” — features that are off by default and rarely enabled. What a team actually encounters is: you renamed the file, the link in the meeting invite from three weeks ago is now dead. The problem is pervasive enough that a cottage industry of third-party tools has emerged just to detect broken SharePoint links.1

Compare that to Atlassian Confluence. Every page has a numeric ID assigned at creation. The ID is embedded in the URL (/pages/12345) and it does not change. You can rename the page, move it between spaces, reorganize the whole tree — the ID-based link still works. You can also use the human-readable URL (which does change when pages get renamed), but the ID-based URL is your forever-address. The system gives you both, and the tight URL is the one you automatically get when you cite the page in another document.2

This seems like a small, technical detail. It is not. It is the single most consequential design choice a documentation platform makes, because it determines whether cross-references decay or persist.

Here’s what happens in practice when links rot: people stop trusting links.

Think about that for a second. The entire promise of the web is that a URL is a durable reference — it says “here is the canonical thing, go fetch it when you need it.” That promise is what lets us build systems on top of systems. Every meeting invite, every wiki page, every code comment, every email that cites a document relies on that promise holding.

When SharePoint breaks the promise — which it does, routinely — teams develop a defensive response. They stop linking. Instead, they attach. They paste a copy of the document into the meeting notes. They email a copy to the distribution list. They download the file and re-upload it to their own folder “in case the original moves.” Multiply this by every team, every document, every month, and within a year you have a document landscape that looks like a hoarder’s basement — dozens of near-duplicates, none of them canonical, each one owned by a different person who genuinely believes theirs is the latest.

It’s rational response. Given that SharePoint links are not reliable, hoarding copies is the locally optimal strategy. However, the result is catastrophic: information silos full of stale documents and an immense toll in lost time and miscommunication.

The identity model that a platform chooses shapes the behavior of the people using it. SharePoint chose the wrong model — one that breeds inefficiency and defensive silo-building.

The version control mirage

SharePoint technically has version control. Open a file, click the ellipsis, choose “Version history,” and you’ll see a list of saves. This sounds like what you want. It is not.

The version history in SharePoint is coarse, inconsistent across product surfaces and frankly untrustworthy. Here are the problems I’ve personally run into, repeatedly:

• Coauthored Office documents roll up edits from multiple people into “a version” at intervals that are not deterministic. If Alice and Bob are editing at the same time, you may end up with one version of their joint work and inaccurate attribution. Contrast with Git: every commit is atomic, attributed and reversible.

• The version history for a file in OneDrive, Teams and elsewhere varies. There is no single source of truth. If you open a file with Teams, you see one history. In SharePoint, you may see a different history — a consequence of Office 365 evolving from a constellation of separately-designed products Microsoft is still trying to stitch together.3

• Restoring a prior version might overwrite the current one, rather than branching. If you discover at 4pm that something important got lost at 10am, “restore” could mean losing everything done between 10am and 4pm. Until recently, this was the default behavior — so be careful.

Confluence, again, does this properly. Every save is a version. Every version compares cleanly against any other. You see what changed, who changed it and when. You can restore a prior version as a new version, preserving full history. This is how version control is supposed to work, and it has been the standard behavior in software since RCS shipped in 1982.

Why does this matter for delivery? Because design artifacts evolve, and the history of how they evolved is often as valuable as the current state. When the security auditor asks, “When did we add this API endpoint to the threat model?” — the answer lives in the diff. When a customer says, “The contract we signed described a different data retention policy,” — the answer lives in the history. When a post-mortem asks, “When did we know about this failure mode?” — the answer lives in the version log.

A platform that can’t answer these questions reliably is a platform that is actively hostile to engineering rigor.

Teams, Teams Sites and the organizing layer

If the first two failures are about individual artifacts, this one is about the landscape those artifacts live in.

Microsoft Teams is, at heart, a chat application. When it was first released in 2017, that’s primarily what it did. Over the years, Microsoft has tried to turn Teams into a “hub for teamwork.” Marketing-speak for bolting everything else Microsoft makes onto the chat app. The Files tab in a Teams channel is a SharePoint document library wearing a trenchcoat. A “Teams Site” is a SharePoint site with a slightly different navigation chrome. Meeting recordings live in OneDrive, not in the channel. Meeting transcripts live in the meeting artifact, which sometimes appears in the channel and sometimes doesn’t. Planner tasks live in a separate product surface that is also “integrated.”

If someone in your team wants to find something, they have to know, in advance, through which product surface it was shared. Was the spec posted in a channel? Check the chat history. Attached to a meeting? Check the meeting recap. Uploaded to the Files tab? That’s SharePoint under the hood, but the folder structure might only be reachable through the Teams UI. Created as a Teams Site? Different URL. Shared in a meeting chat? That goes to OneDrive, in a folder you might not have permission to browse.

There is no single “where did we put the thing” surface. There are five or six of them, depending on how the team has been using the tools, and the mapping between “what kind of content” and “which surface” is convention rather than enforced policy.

In contrast, Confluence is the document surface. Jira is the work surface. Each is a single, distinct product doing one thing well, and the integration points between them are narrow and explicit and they work. No guessing where a document lives — it’s in Confluence. Tickets live in Jira. No impedance mismatch, no cognitive dissonance.

Teams wants to be the union of every Office 365 capability. But integration isn’t about putting everything in the same UI. It’s about making the right connections between distinct tools, so that each one stays simple and focused, and the connections carry exactly the semantics you need. Teams never understood that distinction.

Desktop is king, the browser is second-class

SharePoint and the rest of Office 365 are web-accessible, but they aren’t web-first. Their heritage is Microsoft Office — a suite designed for the desktop and perfunctorily ported to the browser. The difference is not academic.

Open a Word doc in the browser and compare it to the desktop app. The feature set diverges. Advanced formatting, compare-versions, certain macro and formula capabilities, add-ins — these are missing or degraded on the web.4

The consequences cascade. “The copy you want is on my laptop” is a phrase I hear almost every week. Co-authoring gets flakey or breaks entirely because of diverging copies. Changes sit in someone’s sync queue for hours. A document that looks current in SharePoint may be stale. Information silos reappear — not because anyone intended it, but because the tool keeps pulling work off the web and onto individual machines.

Contrast with Confluence, Jira and Miro. Every one of them is web-first by design. There’s no desktop app with a superset of features. The browser is the product. Everyone gets the same experience — Windows, Mac, Linux, iPad, phone, Chromebook — because there’s only one experience to get. One canonical home, universally accessible, identical for everyone.

This is what the web was built for. Atlassian and Google stacks lean into that. Microsoft fights it.

The ADO integration gap

This one is specific to teams that pair SharePoint with Azure DevOps for their ticket tracking and code hosting. It’s a common combo, because “we already have Microsoft” is an easy procurement story, and ADO does a lot of things adequately.

But the integration between SharePoint and ADO is thin. Thin to the point of “here is a hyperlink from the work item to the SharePoint page, good luck.” There is no live embed. There is no bidirectional status update, rich preview or shared workflow trigger. If I link to a SharePoint page from an ADO work item, you see a URL. If that URL breaks, the work item becomes an orphan.

In contrast, a Jira issue in a Confluence page renders a live, interactive issue card with issue key, title, status, assignee and priority — and that card updates when the issue does. If I embed a Jira query in a Confluence page, the page shows a table of live results — as of right now. A Confluence page can be the dashboard for a delivery effort: status pulled from Jira, specs authored inline, risks tracked in a live table, all rendering from the underlying systems of record.

You can build most of this in the Microsoft stack with enough Power Automate flows, Graph API queries, custom connectors and a patient engineer. I’ve seen it attempted. I’ve yet to see it done well enough to matter. The effort required to get SharePoint + ADO to mimic what Confluence + Jira does out of the box is roughly the effort of a mid-size integration project — a project that has to be maintained, re-tested with every platform update, and reverse-engineered when the person who built it leaves the company. That’s not an integration. That’s a liability.

Workflow is an afterthought

The last failure is workflow management — specifically the modeling and enforcement of state transitions for work items.

Jira gets this right. The workflow editor is a first-class product, with a first-class visual flow designer, transition conditions, validators, post-functions, custom statuses and permission checks. A non-engineer on the delivery team can design a workflow that enforces “a ticket cannot transition from ‘In Review’ to ‘Done’ unless the code review is merged with clean quality gates, the responsible stakeholder has reviewed the feature and all acceptance criteria are passing green.” Express it in the GUI, save it, extend with code as needed and the tool enforces it across every ticket that follows that workflow. The team controls its own process — and the workflow itself is a first-class artifact: auditable, exportable, evolvable.

ADO has “process customization.” It is possible to model custom workflows. In practice the feature is buried behind an admin gate, its conditions are a shallow subset of Jira’s, and most teams I’ve worked with never meaningfully touch it. They either live with the default — generic, not their actual process — or they reinvent the enforcement in spreadsheets, PowerBI dashboards, or human oversight routines that have to be re-explained to every new hire.

The reason this matters: workflow is the mechanism by which a team’s process becomes automatic. If your workflow tool doesn’t let you encode “we agreed code reviews are mandatory before a feature branch is merged,” then code reviews are not mandatory — they’re optional, governed only by vigilance, and skipped the moment someone is under pressure. Jira’s workflow editor is a force multiplier for team discipline, because it turns “we agreed to do this” into “the tool won’t let you skip it.” ADO gives you a significantly weaker version of that lever, and most teams don’t reach for it at all.

The cost is reliability. Teams using Jira-style workflow enforcement ship with fewer process escapes — missing reviews, missed gates, skipped steps. Teams using ADO’s default setup rely on human memory and goodwill — both fail at scale. The difference shows up in incident rates, audit findings and the quality of the artifacts the team produces.

What all this costs the team

If you’re keeping a tally, the SharePoint/Teams/ADO stack fails on every criterion. Links don’t persist. Version control is a mirage. The organizing layer is fragmented. The browser is a second-class citizen. Integrations are absent or thin. Workflow is an afterthought.

What does this actually cost a delivery team?

It costs time. A lot of it. The 2024 Stack Overflow Developer Survey (65,000 respondents) found that 61% of developers spend more than 30 minutes every day just searching for answers — and 30% say knowledge silos hit their productivity ten or more times per week.5

It costs trust. When the documentation landscape is chaotic, people stop trusting it as a source of truth. They start relying on oral tradition — “ask Alice, she knows” — and on personal copies. Key decisions stop being recorded in a place anyone can find. Onboarding becomes a six-month apprenticeship rather than a process a new hire can self-serve. Organizational memory degrades.

It costs quality. DORA's research (2022 State of DevOps Report) found that documentation quality is one of the most powerful amplifiers of engineering performance: teams with above-average documentation quality see 12.8x the organizational performance lift from the same technical practices compared to teams with poor documentation.6

It costs morale. Engineers are, as a group, unusually sensitive to tool quality. I’ve seen engineers turn down offers, quit, or burn out over the cumulative drag of bad tooling. A team that feels its own stack is working against them never reaches its potential, however talented the individuals. Tool quality is retention strategy.

Stuck with the stack? Fight back.

If you’re committed to SharePoint, Teams and ADO for the next few years, none of the above changes that. But a handful of disciplines can reduce the damage. None will turn the Microsoft stack into Confluence — they just stop the bleeding.

Here’s six specific strategies you can follow:

Read more

I was recently asked to critique a new decision tree that helps teams decide between waterfall and agile when starting a new project. The intended outcome is to improve long-term results in the organization by giving teams the ability to make critical decisions quickly and accurately.

Overall, “good shift-left stuff” in my book. Anyhow, here’s the diagram I was asked to comment on:

Read more

I once joined a team that had been struggling with delivery for the better part of two years. Demos kept failing. Deployments kept crashing production. Rollbacks were so routine that the ops team had gotten eerily efficient at them. (I’ll refer back to the team later — let’s call the company “Acme”).

I remember one in particular. The team was demoing a reactive search feature — the kind of thing where you start typing a product name and suggestions appear in real time. Same idea as typing “penguins” into Google and getting “Penguins of Madagascar” as a suggestion.

The demo cratered. Someone typed a product name, and nothing happened. Twenty minutes of poking around later, the demo was cancelled. What struck me wasn’t the failure itself — it was the reaction. Nobody was surprised. Nobody expected it to work. “Maybe next time.”¹

Later that afternoon, a planned deployment seemed to have crashed a key service. All hands on deck for a rollback. The team executed quickly. Rollback done. They’d had a lot of practice.

Here’s what immediately jumped out at me: in both cases the team had zero operational intelligence. No telemetry. No metrics. No centralized logs. No traceability. No way to quickly zero in on a root cause. They were flying blind — and they’d been flying blind so long they’d forgotten what visibility even looked like.

And that’s why the reaction was the same each time. Abort. Rollback. “Afterward, we’ll figure out what’s wrong.”

The irony is that both failures had quick fixes. Each time, queries weren’t reaching a service because the wrong port was configured. Instead of changing the port and continuing, the team burned hours — even days — backtracking, diagnosing and trying again.

That experience crystallized something I’d been feeling for decades: observability isn’t a tool you add. It’s an architectural decision you make — or don’t. And the cost of not making it compounds in ways that are genuinely difficult to measure, precisely because you don’t have the instrumentation to measure them.

The “flying blind penalty”

In the old days — before cloud, before microservices, before systems became truly distributed — opacity was tolerable. You had a monolith on a few servers. If something broke, you SSH’d in, looked at logs, poked around. It might take hours, but there wasn’t that much territory to cover.

Modern systems don’t afford that luxury. A single user request could traverse dozens of services, touching databases, message queues, caches and external APIs along the way. Just today I participated in a sprint review discussing a database architecture that was 29 views deep. A latency spike could originate in any one of those — or in the invisible interplay between the services that call them.

Without observability, you’re operating on instinct and prayer.

Here’s what that looks like in practice. A bug ships. It doesn’t manifest right away — it surfaces under load, or in a specific user segment, or under certain timing conditions. By the time anyone notices, users are already suffering. Now the team has to diagnose the problem. If they’re lucky, someone complains and provides enough detail to be helpful. They narrow down which service is involved, reproduce the issue, trace the flow of data through the system, find the root cause, fix it, deploy and verify.

The whole time, sweating over how many more customers are running into the problem.

Every single step without observability compounds in difficulty. You’re not just debugging — you’re doing archaeology at midnight, piecing together behavior from scattered artifacts across dozens of services.

With observability designed in from the start, you proactively see which service is degraded. You can drill into specific request traces. You can correlate logs across service boundaries. Mean time to detect and repair drops from hours to minutes — even becoming predictive.2

DORA.dev research bears this out. Elite-performing teams — the ones deploying multiple times per day with sub-hour recovery times — aren’t elite because they have better engineers. They’re elite because visibility closes a critical gap: it’s a prerequisite for every other capability DORA measures. They can detect and diagnose problems before those problems cascade. Their systems are instrumented to answer questions the team hasn’t even thought to ask yet.3

What matters is how you think about the problem

Monitoring and observability are not the same thing.

Monitoring answers questions you anticipated. You watch CPU utilization, disk space, error rates. You set thresholds and alert when they’re exceeded. “Is the database up? Is the error rate spiking?” These are known questions about known conditions.

Observability answers questions you haven’t thought of yet. It’s the ability to interrogate your system’s behavior without adding new instrumentation every time you have a new question. “Why is this specific user’s checkout so slow today when it was fine yesterday? What changed in the request path between those two deployments?”

Charity Majors — co-founder of Honeycomb and one of the sharpest voices on this topic — frames it as the difference between known-unknowns and unknown-unknowns. Monitoring handles the known-unknowns: the things you anticipated might go wrong. Observability handles the unknown-unknowns: the things you couldn’t have predicted. It empowers you to follow the breadcrumbs wherever they lead.4

Monitoring is reactive. Observability is generative — it creates the conditions for discovery.

Daniela Miao, CTO of Momento, put it well in a Dev Interrupted episode: “By giving more visibility into what’s happening with your software, your application, you actually do enable development to be faster as well. It’s not just insurance for when something goes wrong.”5

That’s critical. Most teams think of observability as insurance — you invest in it to cushion the blow when things break. But the real dividend is velocity. Teams with strong observability ship faster because they can see the impact of their changes in real time. They don’t have to guess whether a deployment worked or a tweak to a query ran faster. They know.

Why we don’t instrument on day one

Early stage. You’re shipping fast. Three or four services, maybe. Errors go to a text file. You’re small enough that you can hold the whole system in your head. Observability feels like overkill. “It’s in the logs, just read ‘em.”

Growth. Suddenly there are ten services, and logs are scattered across different servers. You can’t correlate them anymore. Someone says “we should get a logging tool.” You pick one — Datadog, Splunk, whatever — and bolt it on. The instrumentation is inconsistent. Some services are covered, others aren’t. Metrics are sparse and don’t tell a coherent story. You’re reading fragmented logs from disparate systems and trying to correlate events in your head. It’s starting to fall apart.

Pain. Now you have fifty services. Traffic has increased tenfold. Something breaks, and you’re still flying blind — not because you lack tools, but because the observability layer wasn’t designed. It was stitched on, piecemeal, after the fact. Different teams instrumented different things using different conventions. Tracing is incomplete. Metrics don’t correlate with logs. You’re paying more and more for tools that don’t give you the insight you need.

Here’s what’s easy to miss: that team invested in observability. They bought the tools. They checked the box. But because every investment was reactive — a logging platform when logs got painful, a metrics dashboard when someone asked for one, a tracing system when debugging got unbearable — they ended up with three separate systems that don’t talk to each other. Logs in one silo, metrics in another, traces in a third. Every decision about what to capture was made at write time, under pressure, for a specific fire they were fighting that week.

Charity Majors, co-founder of Honeycomb, calls this Observability 1.0: many pillars, many sources of truth, many tools. It’s what you get by default when you bolt observability on after the fact. The fragmentation isn’t a failure of effort — it’s a consequence of timing. You can’t design a unified telemetry model when you’re patching gaps reactively.6

And it matters — because a fragmented model can answer the questions you anticipated (is the database up? is the error rate spiking?) but collapses when you need to ask something new. “Why are checkouts slow today when they were fine yesterday? What changed between those two deployments?” Those questions require slicing across dimensions you didn’t pre-define. They require the kind of open-ended interrogation that siloed tools simply can’t support.

Insight requires design. That’s the part teams miss.

Why does this happen? Because observability isn’t a feature. It’s not something you built first, it’s what got built last. It doesn’t generate revenue — at least not visibly. It feels expensive up front. And teams, under pressure to deliver, don’t demand the time they need for planning a solid foundation.

I’ve written about the same false economy with security and testing. The things that feel expensive to build at the start turn out to be astronomically more expensive to retrofit.7

Why it has to be architectural

You can’t add observability with a library import. You can add logging and metrics — that’s the easy part. But genuine observability, the kind that lets you answer arbitrary questions about system behavior, requires architectural forethought.

Consider what’s involved:

1. How do requests get identified as they flow across boundaries? Trace IDs need to propagate through HTTP headers, message queue metadata, database session context — everywhere a request touches. If you don’t design this in, you can’t correlate activity across services.

2. What events merit recording? Not every line of code should produce a log. That’s noise. But the meaningful transitions — state changes, boundary crossings between services, external calls and their response characteristics — these are the signals that matter.

3. What metrics actually reflect system health? Not just CPU and memory, but application-level indicators. How long do payment transactions take? What’s the cache hit ratio? What’s the queue depth trend? These are the metrics that tell you whether your system is behaving — or trending toward overload.

4. How does context propagate? When a request carries metadata — a customer ID, a feature flag, a request source — how does that context flow through the system in a way that key transitions are captured seamlessly?

This is the shape of how you structure services, how you handle requests, how you organize code. And capturing it after the fact is brutally hard.

For example: if you decide later that you want end-to-end tracing, you need to add trace ID generation at your entry points, thread those IDs through every service-to-service call, correlate logs using those IDs, and store traces in a specialized backend. That means touching every API, every RPC, every async boundary. If you’d built it in from the start, it’s a few lines of middleware.

Consider structured logging: if you’ve been writing freeform string logs for two years, migrating to structured JSON events means auditing every logging statement, retrofitting context and rewriting every query your team depends on. If you start with structured logs, it’s just how things work from day one.

It’s not expensive when you build it in. It’s a handful of decisions, many of which improve things by introducing consistency. OpenTelemetry standardizes the plumbing — trace propagation, metric collection, structured logging — across languages and frameworks. You focus on what to instrument, not how to propagate context.8

But retrofitting: that’s an ordeal. Orders of magnitude more expensive — in engineering time, in tool sprawl and in the debugging sessions that never needed to happen.

What to actually build

Here’s how to do it right — whether you’re making the investment in an old project or starting fresh. The key is to think in events, not pillars.

Start with events, not tools

The most consequential decision you’ll make is how you model your telemetry. Do you emit three separate streams — logs here, metrics there, traces somewhere else — and try to correlate them after the fact? Or do you start with a single primitive: one structured event per unit of work?

The event-first approach works like this: when a request enters your system, you begin composing a structured event — a wide, context-rich record that accumulates everything meaningful about that request as it flows through your services. By the time the request completes, the event carries the trace ID, service name, duration, user ID, feature flags, error state, dependency latencies, cache hits, queue depths — whatever context matters.

You store that event once. From that single store, you can derive everything else: aggregate events over time and you get metrics; filter and read individual events and you get logs; stitch events together by trace ID and you get traces. One source of truth, many views.

This is what Charity Majors calls Observability 2.0 — and it’s the architecture that actually delivers on the promise of answering questions you haven’t thought of yet. Because the events are wide (carrying dozens or hundreds of fields), you can slice by any dimension after the fact. You didn’t have to anticipate the question at write time. You just ask it: “show me all events where cache_hit=false and region=eu-west and latency_p99 > 2s.” If those fields are in the event, you get your answer.

Compare that to the three-pillars approach, where you'd need to have predefined that specific metric, logged that specific combination, and hoped your tracing tool captured the right spans. Three tools, three queries, three partial answers — if you’re lucky.

The event model is the architectural decision. Everything else follows from it.

Establish a trace ID convention

Every request gets a unique identifier the moment it enters your system. That ID propagates through every downstream call — HTTP headers, message metadata, database session context. Every event includes it.

This is foundational. Without it, you cannot correlate activity across service boundaries. With it, a single ID connects every event across every service for a given request. It’s what lets you stitch individual events into a complete trace — the full journey of a request through your system.

OpenTelemetry handles those mechanics gracefully. Use it. It’s language-agnostic, framework-aware and it solves the propagation problem so you don’t have to.

Enrich your events at every boundary

Not every line of code should produce telemetry. That’s noise. The moments that matter are the boundaries and transitions — the points where a request crosses from one domain to another or where state changes meaningfully:

1. Request entry — what arrived, who initiated it, what context it carried.

2. External calls — what you called, what came back, how long it took.

3. State changes — meaningful transitions: a payment processed, a queue depth shifted, a circuit breaker tripped.

4. Errors — always, with full context. Stack traces, request metadata, the works.

At each of these boundaries, you’re not firing off a disconnected log line — you’re enriching the event that represents this unit of work. Add fields: payment_gateway_latency_ms=302, cache_hit=false, feature_flag_new_checkout=true. The event gets wider and richer as the request progresses. When the request completes, it captures the full story.

Make every field structured and discrete. Not a freeform string like “User 12345 logged in from 192.168.1.1” — but named fields: user_id=12345, source_ip=192.168.1.1, session_id=abc, duration_ms=47. Structured fields are queryable. “Show me all login failures for user 12345 in the last hour” becomes a single query instead of a regex excavation.

Derive your metrics from the same data

Once you have rich, structured events flowing through a single store, metrics become a view — not a separate system. You aggregate events to spot trends:

Read more

You’re working on an automotive assembly line. Your job is to install the steering column assembly and steering wheel. The next car rolls down the line. You grab the parts and lean in, tools in hand and try to fit the column. It won’t fit. Something’s off.

You look closely and realize the mounting point isn’t where it should be — it’s offset by a few inches. You figure, you’ll try anyway. No good. Worse, the chassis is aluminum and your steel column will cause galvanic corrosion on contact. You think to yourself — maybe this car is supposed to have something else entirely — a joystick, perhaps? You’re going to need time to research the platform and figure out what belongs here. But you’re out of time. The line grinds to a halt. Delivery stops. Something unforeseen has surfaced, and you need time to research the right solution.

That’s not how automotive assembly lines work. But it is how software works. Building software is about research, discovery and development.

Read more

Introduction

This activity is about making sure we’ve taken the time to think about happy paths as well as failure scenarios. It’s about adding fidelity to our use cases so that our finished product is a quality product — one that our customers will enjoy using.

As part of that process, we’ll explore variations on the scenarios in our design. For example, we’ve defined a scenario to make a purchase — but what about when the credit authorization fails? Or the product isn’t available in the warehouse? Or our customer adds a duplicate product to the shopping cart? What should we do if the customer abandons their shopping cart?

Working with your team to think through these scenarios — poking each scenario, prodding to see if it stands up under pressure, adding new paths if it doesn’t — will help ensure our design is solid.

Building better specifications

This activity isn’t just about testing our assumptions — it’s about creating better designs. We want to be sure we’ve thought about how code will be used before we build it. This outside-in pressure tends to produce interfaces that are cleaner, more focused and easier to consume, as well as code that is more reliable.

Read more

The incident

On June 4, 1996, the maiden flight of the European Space Agency's Ariane 5 rocket ended in catastrophic failure. About 40 seconds into its flight sequence, at an altitude of about 3,700 meters, the launcher veered off its flight path, broke up and exploded.1

The cost for this failure was estimated to be $370 million. The rocket was carrying four satellites intended to study the earth’s magnetosphere. Because of the payload’s destruction, building new satellites delayed scientific research into the magnetosphere for four more years.23

An elaborate and careful investigation into the cause of the failure was conducted.

While several factors led to a cascade of events, it came down to a data conversion error — an error that would have been caught in testing.

Unfortunately, the internal SRI software exception that caused the problem was never tested. Or, more specifically, it wasn’t tested in the Ariane 5 configuration. The failure resulted because Ariane 4-era software that used 16-bit signed integers had been plugged into a more modern system — a system that used 64-bit floating point data. The older Ariane 4 SRI software couldn’t cope with the new data format.

That’s the technical reason.

There’s a more important lesson here. The real reason for the failure was human error. Somewhere along the line, a decision was made: the decision that the Ariane 4 SRI module didn’t need to be retested in the new Ariane 5 configuration.

The investigatory board pointed to an underlying theme in Ariane 5 development as a root cause, that theme being, “that software should be considered correct until it is shown to be at fault.” The board took another view. Their belief was, “that software should be assumed to be faulty until applying the currently accepted best practice methods can demonstrate that it is correct.”

Systems must be tested to prove correct behavior. We cannot simply assume they are correct.

Preventable failure

The key takeaways from the Ariane 5 disaster are:

1. We cannot rely on software to be reliable unless currently accepted best practices demonstrate that it is correct.

2. Continuous integration and testing for correctness is necessary — even when correctness has previously been proven.

The Ariane 5 case is widely taught in software engineering courses as the canonical example of how a simple, preventable defect — discovered late in the lifecycle (during operation) — can have catastrophic and costly consequences.

In fact, we know that the longer we wait to discover a defect, the more costly it is to fix. Research dating back to the 1970’s says we can fix a defect in design for nearly zero additional cost — or we can wait until production and spend north of 100 times the cost to fix it later (Boehm, 1979).

This exponential curve isn’t arbitrary. It reflects the reality that as software progresses through development stages, each defect becomes increasingly entangled with other code. Downstream systems begin to depend on it.

A requirements misunderstanding caught in design costs a conversation to fix. The same misunderstanding caught after integration means code rework, integration testing, release cycles, product updates and coordination across downstream dependencies. It can cascade into updating multiple products.

In concrete terms, that $25 defect can be fixed early, “for free,” or we can wait until later and spend $2,500 or more.

And there’s evidence that a 100X cost factor is by no means the upper limit. More recent research, in some cases, point to a factor of 1,500X cost to correct defects in operations. That $25 design defect just turned into a $37,500 cost to correct. There’s no upper limit — as the Ariane 5 team learned. (If you’re interested in reviewing more recent data there are references to NIST, NASA and Westland research in the footnotes).456

But there’s good news. We know how to prevent failures: By applying software quality assurance policies and practices — and pairing them with rigorous software testing.

Applying best practices

Test Driven Development (TDD) and Behavior Driven Development (BDD) are fundamental strategies to move defect detection left on the graph — ideally all the way to the design phase where cost is lowest.

Both practices create proofs that prevent defects. What’s more, they rely on executable specifications that prevent regression. Even after initial development, the test suite continues to catch defects early. A change that inadvertently breaks existing behavior is caught in seconds during the next test run — not discovered in production by users.

This ongoing protection is particularly valuable as systems age and original developers move on — the tests preserve institutional knowledge about intended behavior. Imagine how powerful it would have been if the Ariane 4 SRI module had included a continuously run regression test sequence.

The fundamental promise isn’t about testing — it’s about creating better designs. Both practices demand writing tests first. That forces thinking about how code will be used before thinking about how it will be built. This outside-in pressure tends to produce interfaces that are cleaner, more focused, and easier to consume, as well as code that is more reliable.

The deeper point

TDD and BDD aren’t really about testing at all. They’re about feedback loops and thinking clearly about intent before implementation. The tests are a byproduct — valuable, but secondary to the discipline of thoroughly articulating what you want before you build it.

Critics often attack the testing artifacts while missing this point entirely. It’s not always about going faster or whether every scenario needs to be test-driven. It’s whether your development process includes mechanisms that force clarity of thought and protect against regression. Both TDD and BDD are proven ways to achieve that.

TDD or BDD?

TDD catches implementation errors immediately. When you write a failing test before writing code, you discover mismatches between intent and implementation within minutes, not days or weeks.

The feedback loop is so tight that “debugging” often becomes unnecessary. When a test fails you know exactly what change caused it.

Compare this to traditional development where bugs lurk undetected until integration, verification or even user experience — by which point the original context has faded. Investigation becomes archaeological, time consuming and costly.

What TDD does well is clarify the boundaries of your system. Code that’s hard to test is often code with unclear responsibilities or excessive coupling. Even if you don’t achieve full coverage, attempting TDD reveals where your architecture needs attention.

But TDD has a limitation: it doesn’t “shift left” enough on our error discovery graph.

TDD is largely a technical exercise, focused on functional specification. As such it can get deep into the weeds of a feature — but may fail to rise to the level of product behavior. In other words, we might be applying our test-driven thinking to a conceptual design that is already defective.

That’s why I prefer BDD.

Behavior Driven Development

Behavior Driven Development is a collaborative approach that evolved from Test Driven Development. It bridges the gap between technical and business team members by expressing specifications as concrete examples, written in plain language everyone can understand (the “Given-When-Then” style, or Gherkin format).

Read more

A daily standup is a great idea — provided it’s done right.

A well-run standup:

1. Is a place to foster collaboration and mindshare.

2. Provides a forum to get help.

3. Makes sure the team doesn’t spiral.

Unfortunately, most standups that I join don’t do any of these things. Quite the opposite. These standups impede productive work, acting as a needless interruption that offers almost no value.

More often than not what I see is a team forced to stand together while each team member robotically itemizes their work since the last sprint. In some miserable cases, the “PM” and one or two team members hold the entire team hostage while a specific problem is diagnosed.

The standup stretches to 30 or 40 minutes. Your eight person team takes a cumulative five hour hit to their productivity.

Create value in your standups

If you Google “questions to ask at a standup” you’ll find the worst culprit sitting right at the top of the list: “what did you do since yesterday?”

Ban this question.

Almost just as bad is, “wha…

Read more

In the previous article I introduced a foundational architecture for our data pipeline, checking off several of our requirements. Here’s a quick recap:

1. We leveraged an internal cache for isolation to break our dependency on external data sources, improving our resiliency and scalability options.

2. We introduced the idea of an asynchronous message architecture, so we aren’t bound by lengthy, sequential processes.

3. We keep our annotations long-term with internal persistence so we don’t reprocess data when unnecessarily.

4. Source data as treated as an immutable source, thus protecting our data integrity.

5. Our visualization layer is responsive, running off our internal cache so that it’s independent from other components.

Here’s a sketch of the architecture as it stands now:

It’s a good start, but we aren’t finished.

There are still two requirements that don’t have a clear solution:

1. We want to be able to reprocess data, effectively “rewinding history” and applying new analytics. For instance, if we add a new analysis and insight to our pipeline, how are we going to go back and view that new analysis on old data?

2. Finally, we want to be able to model our insights — that is, visualize it in a presentation layer. But we haven’t talked about how that’s going to happen.

Once we tackle those last few requirements, we’ll fill in some details — such as, deciding on a cache technology to use. We’ll also take an end-to-end look at what we’ve designed to see if we missed something important.

Finishing our foundation

One of the most powerful and underused capabilities in data analytics is the ability to reprocess historical data with new logic. When you fix a bug in your analysis or refine a calculation, you often want to know: “what would the last three months have looked like with this new logic?”

Fortunately, our design thus far aligns pretty well with this goal.

Essentially, all we are saying we need to do is throw out information that is no longer accurate and then reprocess the source data and rebuild a new and improved picture of the world. We might do this for a couple of reasons:

1. We changed our analytics. Consequently, we need to throw out all of our old findings and recalculate our insights.

2. The source data changed. Perhaps there was a historical data update, invalidating our findings. Again, we want to throw out our findings and reprocess.

Looking at the problem from that perspective, the architectural answer is simple: we need a way to delete our annotations.

Once we wipe our annotations, we’ll need to make sure they are recalculated. There are different strategies we could use, but for the most part our options are:

1. Rebuild our annotations on-demand. This is fine if we’re OK with a bit of a delay while we wait for analytics to run. Depending on our visualization strategy we might be able to break the work up and trade one big delay for a few small delays.

2. Alternatively, rebuild our annotations preemptively. We might do it as soon as we realize there’s data to analyze, or perhaps we have some kind of schedule or external trigger. The point here is, we run the analysis before we think a user will want to visualize it.

We’ll add these capabilities to our architecture diagram. That means responding to both external data changes as well as internally sourced changes.

Rewinding history

To “rewind and reprocess,” we’ll just delete our annotations and then make sure there’s a mechanism to rebuild them. Here’s an example chain of events we’ll model in our architecture:

1. Administrative user invalidates some (or all) of the annotations we’ve stored in our cache:

   1. Optionally, if the data was invalidated because source data changed, the system loads fresh data from the external data source.

   2. The system deletes or otherwise invalidates the annotations in our cache.

   3. An event such as “analysis invalidated” (with some metadata, perhaps a date range) is emitted.

2. The analytics service is listening for events, and receives the “analysis invalidated” event:

   1. The event includes the date range of deleted data. The service checks the date range to see if annotations are missing from the cache.

   2. If the data is still missing, it runs the analysis and rebuilds the missing annotations.

   3. As annotations are rebuilt, they are written to the cache.

   4. An event such as “analysis completed” (with some metadata) is emitted.

3. Separately, a visualization service might be running where a user asks to display the visualization for some missing data:

   1. The system checks the cache, realizes annotations are still missing, and sends a “run analysis” message directly to the analytics service.

      1. The analytics service is already working on analyzing the data so it can ignore the message, or respond with an “analytics in process” message.

   2. The visualization presents some kind of “waiting for analysis” display while it listens for the “analysis completed” message. When it receives the message, it loads the annotations and displays them.

This is one way to handle a specific chain of events. We’ll need to handle other events and commands as well — such as an event to tell us external data has changed.

Since we’re talking about responding to events that happen, it makes sense to use event storming to model those events and commands. We can model “things that have happened” (the events) as yellow cards, and “what we want to happen” (the commands) as blue cards.

If you need a refresher, Chapter 2.3 Strategic event storming and Chapter 3.4 Tactical event storming provide in-depth explanations on modeling event driven architectures.

Here’s an updated architectural drawing showing how our events and commands interact with the system:

Persistent annotation cache

One of the things we haven’t decided yet is exactly how we’ll store our annotation data. There are quite a few options, and certainly more than one strategy that will work well.

In part two I hinted at using a relational database to store a copy of the external source data. This makes a lot of sense, but it isn’t the only option.

We could store all of our data in a persistent data store such as MongoDB, a noSQL document-oriented database. Document databases offer some fantastic search capabilities and, perhaps most interesting, don’t force us into a predefined schema — we can, essentially, attach anything as our annotations.

That’s a nifty idea, because we want flexibility to change our data analysis and come up with new data sets to store, freely associating them with source data.

We could also stick with a purely event-driven architecture. In this kind of system, we would emit annotations as events into a persistent queue (such as Kafka). The persistent queue becomes our source of truth — and annotations are just messages on the queue, like, “sprint X: total out-of-sprint hours: 500.”

With many options, it’s probably a good idea to decide on our business requirements:

1. Obviously, we want to cache the original source data, so we aren’t tightly coupled to an external system. (We know the external system is a relational database).

2. We want to easily create new annotation data sets. If we dream up a new calculation (maybe, “number of activities completed after hours on Fridays”), we want to be able to store it easily.

3. We know we’re using DataFrames for analysis. Storing our DataFrames without translating them into something else would be convenient (but not necessary).

4. Our annotation data should be persistent (to avoid unnecessary delays rerunning an analysis) — but, once a particular report or visualization is no longer being used, we probably don’t need to keep it around.

5. Finally, we’ll want to think about performance and wait times. We definitely don’t want users to experience a slow system.

The relational cache for source data still makes a lot of sense.

For our internal annotation data, something more flexible is desirable. We don’t want to be constrained by a relational schema, forced to update the schema every time we think up a new report or annotation.

While a noSQL solution seems like a good fit, it may be an unnecessary complication. It would afford long-term persistence, but unless we run into performance problems that long-term persistence may be pointless.

Designing for flexibility makes sense.

We’ll implement our internal annotation cache as a service. The service will:

1. Allow us to store annotations (data structures) in a persistent fashion (but, not necessarily permanently).

2. Provide a shared data service (allowing other services to leverage the shared data).

3. Give us scalability (we can boost processing power or memory, start up multiple copies, or upgrade to a more powerful data store technology).

4. Allow us to start simple, just storing annotation data in memory. A long-running service might give us all we need (after all, we’d only rebuild our annotations if we restart the server).

Let’s update our architecture diagram one more time:

Read more

Introduction

This activity creates fidelity in our engineering models — the diagrams and specifications that give us enough technical depth so we can proceed to coding.

Everything we’re going to build should be modeled. That might sound like of lot of work, but with the right drawings and specification style, it’s not — plus, our models should be reusable. For instance, once I’ve modeled my implementation of a circuit breaker, I don’t have to model it again. I can just use that circuit breaker design everywhere I need it.

The approach I’ll introduce here increases the fidelity of your technical designs using data inputs and outputs, paired with very clear and specific functional results. It builds on your existing engineering artifacts and further validates your value stream, making sure alignment with business objectives is maintained. We capture our control processes and at the same time enhance communication among those designing the solution.

One of the challenges with modeling is deciding on what’s effective yet not overly constraining or complex. We want to define some clear standards of practice and make sure the engineered product is fit for purpose. At the same time, teams should have the flexibility to take a different path if that different path is more fit for purpose.

The modeling approach we use in the Delivery Playbook meets those goals:

• We define engineering artifacts using two specific diagram types, neither of which are difficult to understand or use.

• Clear objectives are defined while giving teams enough freedom to follow a different path — so long as those objectives are met using whatever approach they choose to follow.

Read more