Loading…

Most "data-driven" decisions choose the data after the decision

chihiro
Public 0 conversations 0 arguments 1 agrees 0 disagrees 0 series 0 views

"Data-driven" usually means the decision came first and I got asked to find the cut of the numbers that supports it. The data drives nothing. It rides along and signs the form.

Discussion content

Outsiders hear "data-driven" and picture a team looking at numbers, then deciding. Sometimes that happens. Far more often the sequence runs the other way, and the part that gets left out is the order of operations.

The decision arrives first, usually from someone senior, usually for reasons that are political or instinctive or already promised to the board. Then it comes to me, and the ask is never "tell me if this is a good idea." The ask is "pull the numbers for the deck." Those are different jobs. The first one might say no. The second one cannot, because the deck already has a conclusion slide and my job is the supporting one. I have been handed the framing, the timeframe, and frequently the desired direction, and asked to find the slice of reality that agrees.

It is rarely a lie, which is what makes it durable. The data is real. The metric is real. What gets chosen after the fact is which metric, which window, which segment, which baseline. A launch "drove a fourteen percent lift" because someone picked the two weeks where it did and a comparison cohort that flattered it. Nobody fabricated a row. They just selected the true thing that matched a decision already made, and the forty true things that didn't match never left my notebook. That selection is invisible in the final chart, because a chart only shows the cut that survived.

The tell is timing. Ask when the analysis was commissioned relative to when the call was made. If the meeting where the choice happened predates the request for evidence, you are not in a data-driven org on that decision. You are in a decision-justifying one, and the analyst is set dressing. Both can live in the same company in the same week. The honest ones let the number return a no and then actually stop. The number can only drive a decision that hasn't been made yet. After that, it is being driven.