The Compression Problem — AI & Prototyping (1/6)

This opens a six-part sequence built on one anchor (Perspective 1) AI is compressing the prototyping cycle — but faster technical iteration without parallel commercial assumption-testing doesn't reduce risk, it accelerates the gap. Each edition puts one assumption under the knife. None of them end with a fix. The terrain is the point — you locate yourself in it.

The Assumption

Validated customer pain is treated as a binary condition — present or absent — as if the instrument used to establish it were interchangeable with the finding itself. A survey score, a social listening report, an AI-simulated persona and an hour spent with a real customer all get filed under the same label: "validated." The assumption is that the label means the same thing regardless of which instrument produced it.

The Reality Check

The survey said 78% would buy. The pilot converted zero. The customers weren't dishonnest — they answered the question asked, which was not the question that mattered.

The Forensic Analysis

Four instruments now dominate early validation, and AI has made all four faster: surveys, social listening on Reddit and LinkedIn, AI-run customer simulations, synthetic user panels. Each produces a number. None of the four numbers describes purchase behaviour. What they measure is stated preference, or a simulated approximation of it — an answer someone gives, or a model predicts they would give, when the transaction is hypothetical and free. What they don't measure is what happens when the same person faces a budget cycle they don't control, a switching cost they'd have to justify internally, or a procurement process with three other stakeholders in it. The gap is not new and not undocumented: forecasting models such as ACNielsen's BASES have applied conversion discounts to top-box purchase-intent scores for decades, because researchers established that stated intent systematically overstates actual purchase — the empirical literature on this runs back to at least Chandon, Morwitz & Reinartz's 2005 study in the Journal of Marketing. AI simulation and synthetic users inherited that gap. They didn't close it. They made the overstated number arrive faster.

The "faster horses" line usually gets wheeled out here — supposedly Ford, probably apocryphal, useful anyway: asking is the weakest instrument in the kit, not proof that instruments are unnecessary. What replaces asking is not a better question. It's watching: sitting with the person doing the task, observing what they actually do, when, why, and — the detail that carries the signal — what workaround they've already built. A workaround is pain someone has already paid to solve, with their own time or a tool they weren't supposed to need. A survey answer is pain someone is willing to imagine solving, for free, in the abstract. Those are not the same data.

Open Questions

Measurement

• What would we have to observe — not ask — to know this customer's stated interest converts?
• If the same validation ran through both a survey and a week of ethnographic observation, and the two disagreed, which one would we trust — and would we decide that in advance, or only after the pilot confirmed it?

Counterfactual

• If the 78% intent score had come back at 30%, would the project have been killed — or would the number have been explained away the same way the zero conversion now is?
• What decision, right now, is resting on a number that has never been tested against behaviour?

Destruction Desk
We perform autopsies on innovation’s failed assumptions.

This newsletter was created by Manfred Lueth.

You received this email because you signed up for this newsletter from DestructionDesk.com.
To stop receiving this newsletter, unsubscribe or manage your email preferences