The room is pricing the climber. Every argument about where AI goes next is, underneath, an argument about the climb, recursion, self-improvement, systems that design better systems, the curve bending back on itself and accelerating. AI building AI. The intelligence explosion rendered as an engineering schedule. The progress beneath it is real, and the excitement is earned.

But a climber is only as good as the hill. Optimisation power, however vast, however recursive, is worthless without something faithful to climb toward. The thing nobody is pricing, in all the talk of compute and capability, is the hill itself: the target, the gradient, the signal that tells the system which way is up. We have built an extraordinary engine for going up. We have said almost nothing about who decides where up is.

Take the strongest version of the optimist case first, because it deserves it. The argument doing the rounds is that the firm of the future compounds, that human judgment and machine capability form a loop, each pass sharpening the next, institutional knowledge hardening into systems that improve every time they are used. Offload a task and you keep the learning. Workflows become training signal; training signal becomes capability; capability frees people to aim higher. It is a fine picture of accumulation, and on its own terms it holds.

But read it closely and it names its own limit, almost in passing. Without human direction, the same argument concedes, the loop just runs in circles. That line is doing more work than it is given credit for. It is the whole problem, stated and then stepped over. The loop does not generate its own direction. Something outside it has to say which outcomes count, and that something, not the compute, is the scarce part.

Here is where the usual framing goes wrong. We name "undirected" AI as the thing to fear, autonomous, goalless, off the leash. But a goalless optimiser is a contradiction. A system that optimises nothing does not act; it idles. In practice no agentic system ever lacks a fitness function. It always has one, next-token likelihood, a reward signal, a task it is scored against. What it can lack is a faithful one: a fitness function that corresponds to the thing you care about, rather than a proxy that merely resembles it.

So the real axis was never directed versus undirected. It is faithful versus misdirected. Truly undirected AI is not the danger; it drifts, runs in circles, goes nowhere. The dangerous system is the misdirected one, and it is dangerous precisely because it is competent. It optimises at full power toward an oracle you did not author and cannot see, climbing, fast and well, a hill you never chose.

This already has a name: specification gaming, reward hacking. The system satisfies the metric and violates the intent. It wins the game you wrote instead of the game you meant. Every instance is the same gap, between the target you specified and the target you had in mind, and that gap is not a flaw in one model or another. It is the standing tax on optimising a proxy.

Once you see it this way, a pattern everyone has noticed suddenly has an explanation. They call it the jagged frontier: AI that solves a problem at the edge of human ability, then stumbles on something a child could manage. It looks random. It is not. Capability tracks the availability of a faithful, cheap-to-check fitness signal. Where a clean oracle exists, the system flies. Where one is missing, expensive, or gameable, it stalls, not because the model is weak, but because there is nothing trustworthy to climb.

Look at where AI is strongest: mathematics, code with tests, games, metric-driven search. Each hands the system a crisp verdict on every attempt. ASI-Arch can discover novel model architectures on its own precisely because validation accuracy is a clean, near-ungameable oracle it can consult a million times over. Strip that away, ask for good judgment, sound strategy, a safe decision inside a running plant, and the soaring stops. Generation was never the bottleneck. Verification was.

Which reframes the whole enterprise. A verifier is a fitness function, a thing that scores a candidate against a target. The only difference is what it points at: not the outcome, but the conformance, whether what the system did matches what it was meant to do. If faithful fitness signal is the scarce input, then the machinery that produces trustworthy verification is not a compliance cost bolted on at the end. It sits upstream of all the value; it is what the loop runs on. The failure that machinery guards against is the one I have spent a long time circling, under the name ProbDet, probabilistic output trusted as though it were deterministic fact, and closing that gap is, I think, the load-bearing question of the years ahead.

Here I will admit a bias, because it is also a credential. I have spent a career in operational technology, the systems behind plants, grids, pipelines, where the oracle has always been the expensive, scarce thing. In a safety-critical environment you do not get to assume the system did what it claimed; you prove it, continuously, against what it was supposed to do. The rest of the field is now arriving, at speed, at a bottleneck some of us have been living inside for decades.

So the question we keep asking, how fast can the climber go, turns out to be the less interesting one. The capability will come; it is already coming. The question that decides what it is worth is quieter and harder: who supplies the hill, and can you trust it? We are pouring fortunes into the climb. Almost no one is pricing the hill.