Buying an observability dashboard is not the same as knowing your agent is right, yet that is the trade most teams are quietly making. Two independent 2026 surveys show the field stalling at exactly that line: 89% of engineers have wired up observability while only 52.4% run offline evaluations 1, and a separate survey of 250 agencies names evaluation the top deployment blocker 2. The take I want to defend is narrower and sharper than “evals matter.” It is that teams are mistaking the trace for the verdict, treating a dashboard that shows what an agent did as if it certified the agent was right, and the fix is a placement rule, not a sequence: build the eval harness on the same workflow, in the same sprint, as the dashboard.
The gap, measured
The number that should stop you is the co-occurrence, not either figure alone. In LangChain’s State of Agent Engineering survey (n=1340), instrumentation is near-universal at 89%, evaluation is a coin flip at 52.4%, and 32% name quality as their single top barrier to adoption and deployment 1. Read those three together and the story writes itself. Nearly nine in ten teams can see what their agent did. Barely half can tell you whether it was right. And a third of them are standing in front of the exact quality wall that only the second capability would let them climb.
That last link is the part the survey reports but does not connect. Quality is the named blocker for the same population that bought tracing to saturation and skipped evals. A trace is a rich record of behavior: which tool fired, what the model saw, where the latency went. None of that is a judgment. It tells you the agent returned an answer; it is silent on whether the answer was correct. So a team with dashboards and no harness has instrumented its own blindness. It can watch every step of a run that produced a wrong result and see nothing flagged, because nothing in the stack was ever asked to score the outcome.
A second instrument, a different population
The convergence is what turns this from one survey’s quirk into a pattern worth acting on. A separate 2026 study surveyed 250 agencies, a different population reached through a different frame, and it lands in the same place from the ROI side: evaluation and testing is the number one deployment blocker, named by 49% as a top-three obstacle 2. Where LangChain asked engineers what they had built, this survey asked agencies what was stopping them from shipping, and the answer rhymes.
The same study reports that the bottom-quartile agencies, the ones at 0.7x ROI, almost always lack a workflow-level evaluation harness 2. I read this as corroboration of the mechanism, not proof of it. Weak agencies plausibly lack evals and lack senior engineers and lack scoping discipline all at once; the missing harness may be a symptom of immaturity rather than its cause, and I am not going to claim an eval suite alone would move a struggling shop to 1.2x. What the association does earn is weaker and still useful: the ROI floor and the missing harness travel together, which is what you would expect if a harness is the thing that converts a trace into a signal you can act on. The real weight is carried by the convergence itself. Two instruments, operationalized independently, one measuring offline-eval adoption and the other measuring deployment blockers, across two populations that barely overlap, point at the same absent artifact.
Why they converge
The synthesis lives in the gap between the two sources, not inside either. Neither study set out to prove that observability is being mistaken for quality assurance; each measured one slice and moved on. Put them side by side and the shared shape appears. A trace answers “what happened.” Only a harness that scores the trace against an expectation answers “was it right.” The field has bought the first at near-saturation and is treating it as though it delivered the second. That substitution is the actual failure, and it is not definitional hand-waving, because you can see it in the numbers: the population that owns the dashboards is the same population reporting quality as the wall it cannot get past.
The counter, and why it loses
The strongest case against me is a real one, and it is not that evals are unimportant. It is that you cannot evaluate what you cannot trace. Observability is the substrate an eval harness runs on; you need the trace to have something to score. On that reading, observability-first is not a mistake at all, it is a genuine prerequisite, and my whole thesis collapses into either something trivially true (a trace does not prove correctness, which nobody disputes) or something false (that you should skip tracing).
I concede the prerequisite openly, because conceding it is what defuses the objection rather than what sinks the argument. My claim is alongside, not instead of. It never asks you to skip tracing. The problem is not that teams instrument; it is that they stop at instrumenting and read the green dashboard as a quality result. Adoption does not stall before tracing, where the prerequisite argument would predict; it stalls at tracing, one step short of the harness that would make the trace mean something. And the marginal ROI floor is gated by whether something scores the trace, not by whether you own one more dashboard. The prerequisite is true and it is already satisfied. The gap is the next step, and that is the step the field is not taking.
What I do instead
The rule I actually follow is boring and load-bearing: the eval harness ships in the same sprint as the dashboard, on the same workflow, or the observability rollout is not done. I treat a tracing deployment with no eval suite the way I would treat a service with logging and no tests. It is not a finished quality story; it is a half-built one that happens to look complete because the dashboard is green.
In my experience the failure is not that anyone believes a trace proves correctness when you ask them directly. Asked point blank, everyone concedes the trace is silent on truth. The failure is what the tooling makes easy. A dashboard is a purchase; an eval harness is a project. So the dashboard gets bought, the harness gets deferred to the sprint that never comes, and the team operates for months on the comfortable illusion that a system it can watch is a system it has verified. The cheapest way I know to break the illusion is to refuse to call an observability rollout complete until a harness scores the traces it collects. Build them together. The trace is only a quality signal once something is asking whether the answer was right.