Skip to content
Paula Livingstone writing · projects · tools

Writing

Hunting the Steepest Gradient

An aircraft passing overhead led me to a strange thought about distance. Someone can be physically close yet inhabit a completely different world, while a point only ten feet from an airline passenger can be effectively unreachable without life-supporting technology. That observation suggests a broader way of looking at reality: as countless overlapping conceptual landscapes containing gradients of value, knowledge, trust, accessibility, scarcity, risk and possibility. Some of those gradients are unusually steep, where a small change in state produces a disproportionate change in outcome. Those places often resemble what we call opportunity. But steepness alone is not enough. The more useful challenge is finding the consequential gradients: the ones that actually govern outcomes, can be traversed economically, and perhaps have only recently become accessible. AI may not only change those gradients; it may eventually help us discover ones we have never thought to look for.

I was staring at an aircraft the other day, not for any particular reason, but in that slightly zoned-out state where you are looking at something and allowing your mind to wander.

Something about it struck me as odd.

The people in that aircraft might have been only a few miles from me, and normally the people within a few miles of us are, in some loose sense, our people. They experience much the same weather, use the same roads and infrastructure, and participate in broadly the same local economy and institutions. Geography is not destiny, but proximity creates a surprising amount of commonality.

The people in the aircraft were different. They were physically close, yet they were not really here. They might have come from hundreds or thousands of miles away and be travelling somewhere equally distant, and for a few seconds our coordinates simply happened to converge.

That seemed like one kind of gradient: not physical distance, but something closer to embeddedness. Then another thought occurred to me.

Imagine being inside that aircraft at 35,000 feet. There is a place perhaps ten feet from your seat which you cannot occupy unaided and survive for long. Inside the aircraft you are warm, breathing normally, perhaps drinking coffee and complaining about the Wi-Fi, while only a few feet away is an environment fundamentally hostile to you.

Almost no geometric distance separates those two states, yet conceptually they are enormously far apart.

And that started me thinking about gradients.

The world is covered in gradients we don't measure

We instinctively understand physical gradients such as temperature, pressure, altitude and distance, but much of life seems to consist of gradients that are just as real while being much harder to see.

Things move from known to unknown, trusted to untrusted, safe to dangerous, cheap to expensive, abundant to scarce, possible to impossible and accessible to inaccessible. Information becomes understanding, understanding can become action, and something ordinary in one context can become extremely valuable in another.

These gradients coexist, so an object, person, organisation or piece of information can occupy positions on countless conceptual landscapes simultaneously. Most of the time, movement across those landscapes produces relatively modest effects, but sometimes a tiny change in state produces an enormous change in outcome.

That is where things get interesting, because many of the things we call opportunities seem to occur at precisely those points.

Something has little value in one context and enormous value in another, or information becomes vastly more valuable if obtained slightly earlier. A cheap sensor makes an invisible condition observable, a small software change automates hours of human work, something regarded as waste in one system becomes an input to another, or knowledge commonplace in one discipline solves an expensive problem in another.

The recurring feature is not necessarily invention. It is asymmetry: a relatively small transition creates a disproportionately large change in value.

In crude terms: small movement → large consequence.

That is a steep conceptual gradient.

Arbitrage is perhaps the most obvious economic example, because two markets assign sufficiently different values to essentially the same thing that moving between them becomes profitable. Yet the same structure appears in knowledge, technology, regulation, attention, expertise and countless other places.

So perhaps one way of looking for opportunity is simply to look for steep gradients.

Except there is a problem.

The steepest gradient might be completely useless

A cartoon I came across recently illustrates the problem rather nicely.

A dog wishes it could speak human language and, miraculously, acquires the ability. It excitedly tells its owners that when it whines at the dinner table, it wants some of their food.

The owners explain that they already knew.

The joke works because the dog has misunderstood the constraint. It believes an enormous communication barrier separates its present state from its desired outcome, yet removing that barrier changes nothing because communication was never the important gradient.

The humans already possessed the information. The consequential gradient existed somewhere else, perhaps between knowing what the dog wants and being willing to give it what it wants.

A miraculous technological breakthrough crossed the wrong gradient.

And we do this constantly.

Cartoon in which a dog gains the ability to speak to its owners, only to discover that they already understood what it wanted.
Sometimes removing the most obvious barrier changes nothing.

A company thinks it needs more data, but it actually lacks the ability to make decisions from the data it already possesses. A security operation thinks it needs more alerts, yet its real problem is identifying which alerts matter. A management team commissions another dashboard even though everybody already knows what the problem is, while a customer asks for more information when what they actually lack is confidence. Organisations improve communication around problems that everybody understands but nobody has either the authority or incentive to fix.

The visible constraint attracts attention, but the consequential constraint may sit somewhere else entirely.

So the interesting question is not simply where is the steepest gradient?

It is: where is the steepest gradient that actually governs the outcome?

That is a much harder question, but potentially a much more valuable one.

AI is moving gradients everywhere

This becomes particularly interesting when thinking about AI because generative AI has dramatically reduced the cost of moving between certain cognitive states.

An idea can become prose, a question can become research, a requirement can become code, data can become analysis and a concept can become a presentation, often in seconds. For decades, moving across those gradients required substantial amounts of human time and expertise, whereas some can now be crossed almost trivially.

This is one reason the argument about "AI slop" interests me.

Generative AI has made competent-looking cognitive output extraordinarily cheap, so articles, research, presentations, images, analysis and code can now be produced in quantities that would previously have required armies of people.

We tend to interpret this as AI destroying value, but I think something subtler is happening: the gradients are moving.

If producing information becomes cheap, then producing more information becomes progressively less valuable, but human attention has not become infinite. The gradient between information and important information therefore becomes more significant.

If producing plausible analysis becomes cheap, analysis itself becomes less scarce, but the gradient between analysis and judgment becomes more important. If polished material becomes trivial to produce, polish becomes a weaker signal of expertise, so the gradient between plausibility and trust becomes more valuable.

And if machines can answer almost any question almost instantly, another interesting scarcity emerges: the difference between possible questions and questions worth asking.

That last one may prove particularly important.

For much of history, answering difficult questions was expensive, and we therefore built professions and institutions around the ability to produce answers. If answers become extremely cheap, however, identifying which question exposes a consequential gradient may become considerably more valuable.

Technology doesn't just create opportunities; it relocates them

Suppose moving from state A to state B creates £10,000 of value, but historically making the transition costs £20,000. There is no economic opportunity.

Then some technology reduces the transition cost to £200.

Neither endpoint necessarily changed, and the value difference between them may have existed for decades, but the transition has suddenly become economically viable. What changed was traversability.

AI is doing this across cognitive work at extraordinary speed, but the dog problem remains: just because AI makes a transition cheap does not mean the transition matters.

Producing another report for someone who already has too many reports creates very little value, while producing another alert for someone overwhelmed by alerts may actually create negative value. Producing an answer to the wrong question merely allows us to be wrong more efficiently.

So perhaps opportunity discovery has at least three stages: find the gradients, identify which gradients actually control outcomes, and then determine which consequential gradients have recently become traversable.

That seems to me a much more interesting question than simply asking what AI can do.

Gradient hunting

This suggests a methodology I have started thinking about as gradient hunting.

Take a system: a business, factory, market, network, town, dataset, profession, supply chain or organisation. Instead of beginning by asking what product could be built, start by mapping the conceptual landscapes.

Where are the differences in cost and information? Where does trust change? Where does scarcity exist? What remains inaccessible, and where are decisions unnecessarily slow? Where is expertise concentrated? Where is something abundant on one side of a boundary and scarce on the other? Most importantly, where does a surprisingly small change produce a surprisingly large outcome?

Then ask the question the dog forgot:

If I completely eliminated this gradient tomorrow, would the outcome I care about actually change?

If the answer is no, discard it. You have found a gradient, but not the important one.

If the answer is yes, things become more interesting because you can start asking whether it can actually be traversed, what traversal costs, whether technology has recently changed that cost, and why the gradient still exists in the first place. You can ask who benefits if it disappears, who benefits from keeping it, who else can see it, and what happens when everybody notices it.

That last question matters because a gradient that disappears as soon as it becomes visible may be little more than an arbitrage, while one protected by expertise, trust, capital, regulation, network effects or some other barrier might support something much larger.

Machines could hunt gradients too

This is where the idea becomes particularly intriguing.

Humans are limited in the number of conceptual dimensions we can consider simultaneously, whereas machines are not limited in quite the same way. Imagine giving an AI a complex system and asking it not merely to solve a predefined problem, but to generate thousands of possible conceptual dimensions across that system.

It might consider cost, latency, trust, energy, attention, failure probability, scarcity, information, maintenance, regulatory burden, human effort, predictability and accessibility, along with thousands of dimensions that a human analyst might never think to examine.

Then ask it to search for discontinuities.

Where does a tiny change in one variable produce a large change somewhere else? Where are two conceptually adjacent states assigned radically different values? Where has technology recently reduced the cost of moving between them? Where does everybody appear to be attacking one constraint while another constraint actually determines the outcome?

And then comes perhaps the most commercially interesting question of all: which consequential gradients appear not to have been noticed yet?

At that point AI stops being merely a tool for exploiting known opportunities and starts becoming a tool for discovering unknown ones.

That strikes me as a considerably more interesting application of machine intelligence than producing another PowerPoint.

The landscape was already there

The language of landscapes has a distinguished history. Sewall Wright's adaptive landscape used peaks, valleys and slopes to provide an intuitive representation of a complex multidimensional space, and the metaphor proved useful precisely because relationships that were difficult to comprehend algebraically became easier to reason about spatially.1 I am using the landscape rather differently here, not as a model of biological reproductive fitness, but as a way of thinking about countless overlapping conceptual dimensions and the consequences of moving across them.

The aircraft did not create the pressure gradient outside its fuselage; it merely made me notice it. The dog did not lack information so much as misunderstand where the consequential gradient lay, while AI is not simply creating new capabilities but flattening some old gradients, steepening others and making previously inaccessible transitions traversable.

Perhaps that gives us a different way of looking at innovation.

Instead of constantly asking what we can invent, we might ask what gradients already exist, which ones actually matter, which have just become traversable, and which ones we can see that other people have not noticed yet.

There are probably countless steep gradients around us right now, and most will be useless. Some will be impossible to cross, some will lead to outcomes nobody wants, and others will already have been exploited. Yet somewhere among them will be gradients that are steep, consequential, valuable and traversable, while remaining poorly perceived.

Finding those before everyone else does may be a pretty good definition of opportunity.

Notes

  1. Adaptive Landscapes in the Age of Synthetic Biology, Molecular Biology and Evolution. The paper discusses the origins of adaptive landscapes in the work of Fisher and Wright and the use of multidimensional peaks, valleys and slopes as a framework for reasoning about adaptation. ↩