The Line

I didn't expect the first handrail to be a school equation.

y = mx + c

That line had always felt like the simplest possible thing in algebra. Too simple, almost. x changes, y responds, m controls the slope, and c shifts the whole thing up or down. It’s the kind of equation teachers can put on a board because the shape is visible. You can draw it. You can point at it. You can say: there, that’s the line.

The simplicity hides something important. The equation is a little machine for turning an input into an output.

input goes in
something multiplies it
something shifts it
output comes out

Written slightly differently, it starts looking less like school algebra and more like the beginning of a neuron:

output = weight × input + bias

The mapping is almost embarrassingly direct:

input  = x
output = y
weight = m
bias   = c

The weight is the slope. The bias is the offset. That’s the first clean intuition.

A weight tells you how strongly the input affects the output. If the weight is large and positive, the output rises quickly as the input rises. If the weight is small, the input barely moves the output. If the weight is negative, the input pushes the output down instead.

The bias is different. It’s where the system starts from before the input has done anything. It’s the offset, the default lean, the y-intercept.

So a single bare neuron, at this level, is just:

score = weighted input + starting offset

Or, in the form that became the real handrail:

score = weighted evidence + bias

That was the first small unlock. A neuron begins as a scoring rule.

The Skipped Step

Part of my mental block came from the way linear equations are usually introduced.

At school, the path often goes from:

y = mx + c

to:

y = x²

That makes teaching sense. A straight line and a parabola are both drawable on a two-dimensional graph. They’re visible shapes. You can show the difference between linear and quadratic by drawing one line and one curve.

That route quietly skips a more important step:

output = w₁x₁ + w₂x₂ + w₃x₃ + b

or, less abstractly:

output = 4x + 3y + 6z + c

At first glance, that looks as if it has left the safe world of straight lines. It hasn’t. The inputs are still plain first-power terms. There’s no:

x²
xy
sin(x)
e^x

Each input is simply being multiplied by a weight and added to the total.

4x → x contributes with strength 4
3y → y contributes with strength 3
6z → z contributes with strength 6
c  → the starting offset

The arithmetic hasn’t become exotic. It’s still multiply, add, offset.

The picture, though, has changed. With one input, the equation can be drawn as a line. With two inputs, it becomes a plane. With three inputs, it already strains ordinary visual intuition. With more than three, the neat graph-paper handrail evaporates.

That’s where the mental trap sits.

We’re taught to associate “linear” with a visible line on a page. That’s only the one-input version. The deeper idea is:

weighted sum of inputs

That’s the bridge into neural networks.

A neuron doesn’t need one input. It can have many. Each input contributes some amount of evidence, positive or negative, strong or weak, and the bias shifts the baseline.

So the line becomes a ledger.

this input adds evidence
that input subtracts evidence
this one matters a lot
that one barely matters
the bias sets the starting position

That’s still linear in the useful sense. It’s just no longer comfortably drawable.

The Boundary

Once the graph disappears, the useful question changes.

With y = mx + c, the natural question is: What does this look like?

You draw the line. You inspect the slope. You see where it crosses the y-axis.

Once the input space grows beyond what the mind can picture, that question starts to fail. The equation may still be simple, but the geometry has moved out of view.

The better question becomes: Which side of the boundary does this point land on?

That’s the real neural-network handrail.

Take a simple scoring equation:

score = 4x + 3y + 6z + c

If all we do is calculate the score, the neuron produces a number. Compare that score to a threshold, and it starts dividing the world:

score > 0 → one side
score < 0 → the other side

The place where the score equals zero is the boundary:

4x + 3y + 6z + c = 0

In two dimensions, that boundary might be a line. In three dimensions, it might be a plane. In higher dimensions, it becomes a hyperplane, a flat divider in a space we can’t directly visualise.

The word “flat” matters. The boundary hasn’t become curved or magical just because we can’t see it. It still comes from the same kind of weighted sum. The space it divides has simply become too large for our visual machinery.

That changes the intuition.

A neuron isn’t mainly a line.

A neuron is a boundary-maker.

It takes an input point, scores it, and helps decide which side of a learned division that point belongs on.

input point
→ weighted evidence
→ bias shift
→ score
→ side of boundary

That was the second unlock.

We care less about the look or shape and more about how it divides the weird space we know it lives in but can’t see.

The model doesn’t need to see the space. It only needs the arithmetic. Multiply the inputs by their weights. Add them. Shift by the bias. Compare the result. Pass the pattern forward.

Humans lose the picture after three dimensions. The equation doesn’t care.

The Token Machine

That same handrail, weighted evidence, invisible space, boundary-like pressure, survives the jump from a toy neuron to a language model.

The scale changes. The machinery becomes much more complicated. The basic move remains recognisable:

input becomes numbers
numbers are transformed
scores are produced
something is selected

A language model receives text, and the text is first carved into tokens.

text → tokens → token IDs → vectors → layers → final vector → token scores

A token may be a word, a word-part, punctuation, a space-plus-word, a number chunk, or a code fragment. The model doesn’t really predict the next word. That’s the simplified public version. More accurately, it predicts the next token.

So a prompt like How are you? is split into pieces, turned into token IDs, and those IDs are used to look up vectors.

Something like:

"How"  → token ID → vector
" are" → token ID → vector
" you" → token ID → vector
"?"    → token ID → vector

The token ID is an index number. The vector is where the model starts working.

And this is where the earlier dimensionality problem comes roaring back. A token vector isn’t one number. It’s a row of many numbers. In a real model, that can mean thousands of dimensions.

So the model is carrying something more like:

"How" → [0.18, -0.41, 0.07, ...]

That vector is the live internal state. It’s the point in the weird space.

Then the layers begin transforming it.

A token near the start of the process is more like a raw entry from the embedding table. After several layers, it has been changed by context. The token is no longer merely the token. It’s the token-in-this-sentence, token-in-this-conversation, token-in-this-position.

That distinction matters.

The word bank in river bank and investment bank may begin from the same token or similar token pieces, but the surrounding context pushes the internal state in different directions. The model is repeatedly moving vectors around inside a learned space.

That sounds alien until we drag it back to the handrail.

A layer is doing learned transformations of evidence. It takes a current state and applies trained machinery that amplifies some patterns, suppresses others, and shapes what the representation is becoming.

Not in English. Not consciously. Not with labelled concepts neatly written on the neurons.

Structurally, the pattern is still:

weighted evidence → transformed state

The line has disappeared. The graph has disappeared. The arithmetic hasn’t.

The Trained Landscape

At the end of this process, the model has to produce the next token.

Not the whole answer.

Not the full sentence.

One token.

The final internal vector is projected onto the vocabulary, producing a score for each possible next token. These raw scores are often called logits, but the important idea is simpler:

final internal state → giant learned scoreboard → possible next tokens

For a given prompt, the model may assign high scores to some continuations and low scores to others.

After How are you?, some next-token candidates are strongly supported by the trained landscape. Others are barely plausible.

The exact token may not be I’m, because I’m might be split into smaller token pieces depending on the tokenizer. The model might select something like:

"I"

then:

"'m"

then:

" doing"

or it might have a larger token available. The exact split is implementation detail. The important point is that the model selects from possible tokens, not from whole human intentions.

It’s tempting to say that one token “wins”.

That’s useful, but it’s also an abstraction.

Inside the model, there isn’t usually a little tournament where one meaning defeats another. There are activation patterns, score distributions, pressures, and relative likelihoods. At the output stage, one token may be selected, but that selection is the visible tip of a much larger numeric process.

So when we say:

the token wins

we mean:

the trained system assigned this token enough score relative to the alternatives,
and the decoder selected it

That decoder may be greedy, always choosing the highest-scoring token. Or it may sample from the distribution, allowing some controlled randomness. Settings like temperature, top-k, and top-p adjust how tightly the system follows the highest scores.

The model isn’t selecting from raw possibility.

It’s selecting from trained possibility.

That phrase matters.

Before training, the weights are mostly useless. The landscape has no meaningful shape. The prompt falls into noise and the output is noise.

After training, the landscape has grooves, slopes, channels, and basins carved into it by vast amounts of text. Certain contexts lean toward certain continuations. Certain forms invite certain responses. Certain phrases pull the model into certain regions of behaviour.

So when the model sees How are you?, it doesn’t check its wellbeing. It doesn’t inspect its mood. It doesn’t discover an inner state of being fine.

It lands in a region of trained language space where polite conversational continuations are likely.

The response can still be useful. It can still be socially appropriate. It can still help the conversation proceed.

But the source of the response is trained pressure.

prompt
→ activation pattern
→ trained continuation pressure
→ token score
→ selected token
→ repeat

The model has learned an enormous amount about how language behaves. The “winner” is a hard output event produced by a trained numeric landscape.

The Missing Signal

This is where text starts to show its poverty.

Human communication is words, tone, timing, posture, expression, breath, hesitation, shared place, shared history, social relationship, and the emotional weather in the room.

A text-only model gets the flattened export.

It gets:

words
punctuation
spacing
capitalisation
ordering
conversation history

It doesn’t directly get:

intonation
facial expression
body language
hesitation
volume
pace
eye contact
smell
touch
physical situation
the look someone gives before they answer

Take one word: Fine.

That can mean genuine acceptance, annoyance, reluctant agreement, hurt, social closure, quiet resentment, or the beginning of tears.

A human standing in the room gets far more than the word. The tone may carry the whole message. The pause before it may matter. The face may contradict the sentence. The body may tell the truth while the mouth performs politeness.

A text model gets four letters and a full stop.

It has to infer from residue.

That isn’t a small problem. The model is constantly doing interpretation under missing signal conditions. It has to use surrounding text, punctuation, phrasing, topic, prior turns, and learned patterns to guess what kind of human situation the words came from.

Sometimes that works beautifully.

Sometimes it fails badly.

Because text isn’t full communication. Text is a compressed export of communication.

This also explains why models can seem both uncannily sensitive and strangely cold. They can detect many textual signs that humans use to encode emotion, hesitation, irritation, sarcasm, urgency, or grief. But they don’t receive the embodied event itself.

They see the transcript, not the room.

And that matters for the next step, because once we start asking whether a model can answer as if it cares, we’ve to ask what kind of system care actually is.

Is care just the right output in the right context?

Or is care something deeper: a state that modulates the system from inside?

Modulation

This is where the word arrived.

Modulation.

That was the thing I had been circling around without naming.

A trained model has weights. A biological brain has learned structure too. But an organism isn’t just a fixed set of weights receiving an input and producing an output. The whole system is constantly being reweighted by internal state.

Hunger changes what the world means. Fear changes what the world means. Fatigue changes what the world means. Attachment changes what the world means. Pain changes what the world means.

The same signal doesn’t pass through the same system in the same way every time. The system is modulated.

That matters when people talk about models sounding as if they care.

A language model can produce caring language: I’m sorry you’re going through that. That sounds painful. I’m here to help.

Those sentences may be useful. They may even be the right sentences. But care, in an animal, is more than an output form.

I think of my dog.

I love my dog, and I’m pretty sure my dog loves me. Not in the abstract, not as a proposition, not as a sentence-completion pattern. There’s a bodily fact there. A history. A routine. A smell. A face. A presence. A dependency. A mutual recognition.

The thought of harming her is almost impossible to let my mind inhabit. Something in the system revolts before language even has time to tidy it up.

That isn’t simply:

dog = positive concept

It’s more like:

dog-related signal
→ attachment appraisal
→ protective salience rises
→ harm aversion spikes
→ action inhibition strengthens
→ care/protection becomes the dominant tendency

There are learned weights in that. Of course there are. History matters. Repetition matters. Association matters. Memory matters.

But there’s also modulation.

The dog is processed as more than another object in the scene. The dog changes the state of the processor.

That’s the missing layer.

The clean version isn’t a flat list like:

hunger
fear
pain
attachment
hormones
social bonding
reward

Those things don’t all sit at the same level. They interact across layers.

A better carve-up is:

external/social cue
→ appraisal / meaning
→ bodily/autonomic response
→ neurochemical/hormonal modulation
→ action tendency
→ memory update

So “my tribe” and “hormonal state” aren’t the same kind of thing. My tribe is a social reality, or a perceived social reality. The body recognises belonging, safety, threat, loyalty, exclusion, status, obligation. Those appraisals can then trigger hormonal and neurochemical changes.

The tribe may release the happy chemistry. The threat to the tribe may release the fight chemistry. The loss of the tribe may release grief.

The hormone isn’t the social bond. It’s one mechanism by which the social bond reconfigures the system.

That’s what modulation means here.

It is not decoration. It is not emotional colour sprinkled over cognition after the “real thinking” is done.

Modulation is control architecture.

It changes what becomes salient. It changes what is ignored. It changes what is intolerable. It changes what counts as reward, threat, comfort, duty, disgust, risk, opportunity, or home.

A text model can represent the phrase I care about my dog. It can produce the language of attachment. It can infer that a person talking about a sick dog may need gentleness. It can generate the shape of concern.

But it doesn’t have a dog-shaped attractor in an organismic system. It doesn’t have oxytocin, pain, smell, shared sleep, a warm body at its feet, or the fear of a future absence.

It has trained language behaviour.

That may still matter. It may still help. It’s just not the same class of thing.

Care as output behaviour isn’t the same as care as internal condition.

And once that distinction is visible, the whole question of artificial general intelligence changes shape.

There’s a hint here of the larger destination. If intelligence is always shaped by the system that carries it, then intelligence may never be a mirror floating outside the world. It may be something more local, more partial, more embodied: a place where the world begins to model itself from within.

The Organism

A maximally scaled language model may climb one cognitive ladder very fast.

That ladder is real. It isn’t nothing. It may include abstraction, translation, coding, summarisation, analogy, planning in language, and a kind of uncanny compression of human textual experience.

But an organism isn’t one ladder.

A biological mind isn’t just a language engine that got big enough. It’s a layered survival machine. It has a body, drives, thresholds, fatigue, pain, fear, appetite, attachment, immune state, threat response, social needs, reproductive pressures, and action consequences.

The language model loop is roughly:

text
→ activations
→ token scores
→ output token

The organism loop is closer to:

world/body/social input
→ appraisal
→ modulation
→ perception/action bias
→ behaviour
→ consequence
→ bodily/social feedback
→ memory update

Those are very different systems.

An LLM can describe hunger. It can reason about hunger. It can predict what hungry humans might say. It can write a recipe, explain blood sugar, or produce a poem about famine.

But hunger doesn’t narrow its attention from inside. Hunger doesn’t make food cues glow. Hunger doesn’t make delay intolerable. Hunger doesn’t bargain with dignity, impulse control, memory, irritation, and the smell of bread.

A human can decide intellectually that a thing doesn’t matter, while the body insists that it does.

That isn’t a bug in biology. It’s part of the architecture.

The same applies to fear. Fear isn’t merely the sentence I’m afraid. Fear is a system-wide change in readiness. The world sharpens. Ambiguity becomes threat-coloured. Attention narrows. The body prepares movement before the intellect has finished its report.

Attachment is similar. It isn’t just a belief that someone matters. It’s a persistent modulation of priority. Their pain matters differently. Their absence matters differently. Their danger matters differently.

So when people ask whether LLMs can become minds merely by scaling, the question feels malformed.

Scaling may keep improving the language-pattern engine. It may keep producing better abstractions over text. It may make the system more useful, more fluent, more tool-capable, more agentic-looking, and more able to simulate reasoning.

Scaling the text predictor doesn’t automatically create the rest of the organismic stack.

It doesn’t automatically create:

embodied need
self-maintenance
homeostasis
pain
fatigue
attachment
risk to self
social belonging
action consequence
bodily feedback
modulated priority

Those are different loops.

This doesn’t mean LLMs are useless. That would be stupid. They’re already extraordinary.

It also doesn’t prove that LLMs can never be part of AGI. They may well become central components in larger systems. A language model could be embedded inside an architecture with tools, memory, sensors, actuators, reward signals, self-monitoring, and persistent goals.

At that point, the LLM wouldn’t be the whole mind.

It would be one subsystem inside a larger machine.

That’s closer to the biological analogy. Language isn’t the whole human mind. Abstract reasoning isn’t the whole human mind. Cortex isn’t the whole human mind. A mind isn’t just the clever bit. It’s the clever bit inside a body, under modulation, paying attention to a world that can hurt it, feed it, bond with it, abandon it, or kill it.

A scaled LLM can produce a better and better map of human textual patterns.

An organism has to live.

That difference isn’t sentimental. It’s architectural.

The Interface

Organisms have another bottleneck.

Energy.

A language model in a data centre can externalise the bill. The electricity, cooling, chips, storage, networking, supply chains, and human labour all sit outside the model. The model doesn’t have to feed itself. It doesn’t have to repair its own tissue. It doesn’t have to move through mud, escape predators, carry young, fight infection, sleep, digest, heal, mate, or survive winter.

An organism carries the bill every second.

That changes what intelligence is.

Evolution doesn’t optimise for maximum intelligence in the abstract. It doesn’t reward organisms for seeing reality as it truly is, in some pure philosophical sense. It rewards workable behaviour under constraint.

find energy
avoid harm
reproduce
protect kin where useful
cooperate when useful
compete when necessary
survive long enough

Everything has a cost. A bigger brain costs energy. Better perception costs energy. Longer childhood costs energy. More memory costs energy. More accurate modelling costs energy. Even attention has a cost, because attending to one thing means not attending to something else.

So biological intelligence is a negotiated bargain.

accuracy versus speed
memory versus forgetting
curiosity versus caution
social trust versus suspicion
energy use versus capability
exploration versus safety
individual survival versus kin or tribe

The organism is trying to remain a viable participant in a competitive landscape full of other DNA.

That landscape isn’t static. It contains predators, prey, parasites, mates, rivals, kin, microbes, plants, symbionts, strangers, enemies, allies, and tribes. Every organism is surrounded by other strategies. Some cooperate. Some deceive. Some hunt. Some hide. Some attach. Some infiltrate. Some mimic. Some consume.

So intelligence is strategy under pressure.

And if perception evolved inside that bargain, then perception is unlikely to be a transparent window onto reality. It’s more likely to be an interface.

The gentle version of that claim is easy enough: we don’t perceive all of reality. We perceive what our kind of organism evolved to perceive.

The stronger version is more unsettling: perhaps even the things that feel most basic to us, objects, space, time, cause, distance, separateness, aren’t reality as it’s, but interface elements. Useful icons. Ways for a survival machine to act without needing access to the underlying machinery.^[1]

A desktop icon isn’t literally the file. A folder on a screen isn’t a tiny folder inside the machine. It’s an interface object. It hides almost everything true about the underlying computation, and that’s precisely why it’s useful.

Maybe perception does something similar.

The apple is reality-as-rendered-for-an-animal-that-can-eat-apples.

The cliff edge is reality-as-rendered-for-an-animal-that-should-not-step-forward.

The angry face is reality-as-rendered-for-a-social animal that needs to track threat, status, intent, and belonging.

This doesn’t mean the world is fake. That’s too easy. It means the world we experience may be an action-shaped rendering.

Even when science extends our senses, it still does so through interface layers.

We can name things we don’t naturally perceive:

ultraviolet light
infrared light
magnetic fields
radio waves
chemical traces
microscopic life
atomic structure
air pressure outside our hearing range
neutrinos
curved spacetime
quantum states

That list is still tame.

Those are known unknowns. They’re things we’ve already dragged into the labelled map. They have names, instruments, equations, papers, diagrams, arguments.

The deeper problem is what humans can’t even frame.

What if there are aspects of reality that don’t fit our inherited grammar of:

object
place
property
cause
event
before
after
inside
outside
thing
relation
number
state

What if some of reality is hidden because our categories are wrong-shaped for it?

That isn’t a claim that anything in particular exists beyond the interface. It’s a claim of humility. Our known unknowns are still human-labelled. They’re still made from the conceptual machinery of a primate species that learned to survive on a middle-sized planet, at a middle-sized scale, with middle-speed senses, in a social world of bodies, food, weather, danger, and kin.

The edge of our map isn’t the edge of reality.

It’s the edge of our current labelling system.

The Cave Writing

This loops back to language models.

A language model trained on human text doesn’t escape the human interface. It inherits it. It inherits our categories, metaphors, salience, blind spots, scientific labels, folk psychology, myths, arguments, confusions, and useful lies.

It doesn’t access reality directly.

It accesses human textual traces of human interface-bound experience of interactable reality.

So when a model talks about hidden reality, it naturally lists the hidden things humans have already named. That isn’t the edge of what is. It’s the edge of what has made it into language.

The machine we began to understand through vectors and token scores turns out to be trained on the very interface we’ve just exposed.

It isn’t outside the cave.

It’s trained on cave writing.

The Soup

This is where the whole thing starts to collapse.

Not into nonsense. Not into nihilism. Into something stranger and less comfortable.

We began with a line:

y = mx + c

A clean little model. A handrail.

Then the line became a neuron. The neuron became a boundary-maker. The boundary disappeared into high-dimensional space. The high-dimensional space became a language model. The language model became a trained landscape. The trained landscape exposed the difference between output and inner condition. The missing inner condition led to modulation. Modulation led to the organism. The organism led to energy, evolution, survival, and perception as interface.

And now we’re left with the reflexive problem.

Any intelligence trying to describe reality is itself inside reality.

It doesn’t get to step outside the whole thing, turn around, and inspect it from nowhere. It has to use the tools available to it: senses, instruments, language, mathematics, memory, analogy, models, culture, error correction, and doubt.

But all of those tools are themselves part of the system.

A mind can’t fully step outside its own interface to prove that its interface is reality.

A language model can’t step outside human language to prove that human language has captured the real.

A scientific theory can’t arrive as pure reality; it arrives as symbols, measurements, instruments, assumptions, and predictions inside the world it describes.

This is the soup.^[2]

intelligence inside reality
using symbols shaped by partial access to reality
trying to describe reality

There’s a temptation here to become either cynical or mystical.

The cynic says: if all perception is interface, nothing means anything.

The mystic says: if the interface isn’t reality, I’m free to invent whatever I like beyond it.

Both are too easy.

The harder position is to stand at the edge and not pretend.

The map is not the territory. But the map is not useless.

The interface may not be ultimate reality. It’s still the surface where reality has touched us.

Science doesn’t need a god’s-eye view to matter. Bridges stay up. Medicine works. Rockets are caught by chopsticks. Dogs need fed. Fire burns. Grief hurts. Soup can be hot.

So this doesn’t collapse into “nothing is real”.

It collapses into humility.

No map is final. Some maps are better than others.

Some models compress more usefully. Some predictions survive contact. Some interfaces let organisms live. Some theories carve reality at more durable joints, even if those joints aren’t the final furniture of the universe.

But there’s wonder in the incompleteness.

If our perception is partial, reality isn’t smaller than we thought. It’s larger. If our categories are evolved, then whatever exceeds them isn’t necessarily empty. If even our known unknowns are human-shaped, then the dark around the map isn’t merely ignorance. It’s possibility without handles.

That doesn’t prove anything supernatural.

It doesn’t license fantasy.

But it does open a door to awe.

The boundary of the map isn’t a wall around reality. It’s a shoreline.

By this point I’m no longer looking at the line on the school graph. I’m standing at that shoreline with the same mammalian equipment that brought me here: hunger, attachment, doubt, language, and love.

We may be in the soup, but we can still tell whether the soup is hot.

And perhaps the wonder isn’t that the soup defeats us, but that anything inside it can taste, measure, name, doubt, love, and ask what the soup is made of.

Maybe intelligence isn’t a mirror held up to reality.

Maybe intelligence is one of the ways reality becomes locally aware of itself.

Not completely. Not finally. Not from outside.

Locally. Partially. Under constraint.

Endnotes

1. The perception-as-interface framing overlaps with Donald Hoffman’s work on perception, fitness, and reality. I’m using the interface intuition here, not depending on the full metaphysical programme. ↩
2. “Gödelian” is used as flavour rather than as a formal theorem claim. The analogy is the structural discomfort of a system trying to complete an account of itself from inside itself. ↩