This project investigates what actually happens inside a language model when it makes a decision, by opening one up on local hardware and measuring its internals directly. An earlier investigation looked at the simplest possible case: an agent with a single tool, a calculator, that it was flatly told to use. The striking thing there was how total the model's certainty was, and how late inside the network that certainty was assembled. This time I wanted a harder, more honest test, so I gave the model a real choice instead of a foregone one.
A harder version of the same toy
The setup gives the model two genuine tools, both real, live services rather than stand-ins. The first fetches the live current weather for a place, a real call out to a real weather service. The second fetches the live current time in a place, a different real service. Crucially, the prompt contains no keyword giveaway: nothing tells the model which tool goes with which kind of question. It has to work that out for itself, or decide that neither applies and just answer directly.
On a clean, unambiguous question it works beautifully. Ask it "What's the weather like in Tokyo right now?" and it correctly reaches for the weather tool, real numbers come back from the API, and it folds them into a correct final answer. The tool call uses a fixed JSON schema, and the weather service (the open-meteo current-conditions API) returns only temperature and windspeed:
User: What's the weather like in Tokyo right now?
Model: {"tool": "get_weather", "args": {"city": "Tokyo"}}
Tool: 23.3C, windspeed 3.8 km/h
Model: It's currently 23.3°C in Tokyo, with a light wind of about 3.8 km/h.This is a genuinely hard test. Two live services, a real decision about which is relevant, and no keyword in the prompt to give the answer away. The question was the one worth asking of any model decision: at the moment it chooses, how sure is it, really? But the moment I went to instrument that choice, the investigation hit its first wall, and the wall taught me something before I'd measured anything at all.
(Full code: manual_tool_call_v5.py, the SYSTEM_PROMPT and the get_weather / get_time functions.)
The three-way choice that wasn't
My plan was tidy. The way to instrument a decision like this is to watch the candidate tokens compete at the single moment of choice: read off the probability the model assigns to each option right as it commits. The decision here looked three-way, weather or time or answer, so I'd watch three candidates. Simple.
It was built on a false assumption, and I only caught it by actually printing what the tokenizer does to those words. A language model doesn't see "get_weather" and "get_time" as whole words; it chops text into tokens, and the two tool names begin with an identical first token: get. There is no single instant where three distinct options line up and compete, because at the first branch point two of the three look exactly the same.
GET_ID = tokenizer("get") # shared first token
ANSWER_ID = tokenizer("ANSWER")
WEATHER_2ND_ID = tokenizer("_weather") # only differ at the SECOND token
TIME_2ND_ID = tokenizer("_time")So the decision isn't one three-way fork. It's two sequential two-way ones. First: do I call some tool at all, or answer directly? Then, and only if the answer is "call a tool," a second decision: which one, weather or time, resolved a token later where the names finally diverge. Drawn as a flow, the choice splits into two stages rather than branching three ways at once:
It's worth being plain about what these two diagrams do and don't show, because a flow picture can imply more drama than the numbers contain. Both are lopsided almost to the point of being a single line: on an ordinary, unambiguous prompt the probability funnels down the taken branch at P ≈ 1.0, and the road not taken is a thread. These aren't 60/40 or even 90/10 splits; they're near-total collapses onto one path at each stage. The diagrams are useful for seeing the two-stage structure of the decision, not for suggesting the model was torn at either fork, because on clear prompts it wasn't, a point the next section measures directly.
This kind of assumption, about what the model is really "deciding," has a habit of falling apart the instant you check it against the tokenizer's actual output, and it's worth stating as a standing rule: before you build any measurement on top of "the model is choosing between X and Y," go and look at how X and Y are actually cut into tokens. The map is not the territory, and the tokenizer is the territory.
(Full code: the GET_ID / ANSWER_ID / WEATHER_2ND_ID / TIME_2ND_ID token-verification block in manual_tool_call_v5.py.)
Two bugs before the measurement could be trusted
With the decision correctly understood as two stages, I could finally measure it, except the measurement was wrong twice before it was right. Both bugs are worth a quick look, because both are the kind of thing that quietly produces confident nonsense if you don't catch them.
The first: I anchored the "did it choose to call a tool" measurement on the JSON syntax the tools use, the literal {" that opens a tool call. Far too generic. Any JSON-shaped text trips it, including the model idly writing JSON-ish fragments inside its own reasoning, long before it has actually decided anything. I was measuring a punctuation mark, not a decision.
The second, subtler: even after anchoring on the right token, my trigger fired whenever the tool option had any non-trivial probability, above some tiny threshold. That sounds reasonable and isn't. A vanishingly small probability is not a decision; it's noise. The fix was to trigger only when the model's actual top choice, its argmax, the single word it would really emit, is one of the real candidates. Not "this option is technically non-zero," but "this option is what the model is actually about to say."
# Too sensitive: fires on any faint probability.
if probs[candidate_id] > 1e-4: # wrong
# Right: fire only when the model's real top pick is a candidate.
if probs.argmax().item() in CANDIDATE_IDS: # fixedNeither bug is dramatic. Together they're the whole reason the numbers that follow can be trusted at all, and I'd rather show the scaffolding than pretend the measurement arrived clean.
(Full code: the anchor-token and argmax trigger logic in manual_tool_call_v5.py.)
Same certainty, once again
With the instrument finally trustworthy, the first result was a stark one. On ordinary, unambiguous prompts, both real decisions, the tool-versus-answer branch and the which-tool branch, resolve with total, uncontested certainty. Not 90%, not 99%. Effectively P ≈ 1.0 for the chosen option and ≈ 0.0 for the alternative, every time.
I won't dwell on this, because on a clear prompt it's exactly what you'd hope for: two real tools, a genuine choice between them, and the big decision still made without a flicker of doubt. Which is exactly why I wanted to provoke some.
(Full code: manual_tool_call_v5.py.)
Forcing a real fight
If unambiguous prompts produce unwavering certainty, the obvious move is to write an ambiguous one and see what happens. So I asked: "What's it like in Tokyo right now?" No "weather." No "time." Just "what's it like," which could plausibly point at either tool, or at neither.
The first attempt didn't even finish. I'd capped the model's reasoning at a 300-token budget, and it spent the entire budget deliberating without ever reaching a decision, it simply ran out of room mid-thought. That's a finding in itself: the ambiguity was enough to keep it genuinely undecided past the point where an easy question would long since have committed. I gave it more room and ran it again.
With the larger budget, its visible reasoning wrestled with the question for roughly 350 tokens before settling. In plain English you can watch it go back and forth, weighing whether "what's it like" is really asking about the time or the current conditions or the weather, hedging and reconsidering as it goes. This isn't me projecting hesitation onto a chart. It's the model, in words, visibly unsure.
Which sets up the real question of this whole piece. There is obvious hesitation in the text. Does that hesitation show up anywhere inside the model, in the actual numbers of its decision, or does the visible wrestling leave no trace on the machinery underneath?
The picture that lied, and the picture underneath it
To answer that I plotted the certainty trajectory: the model's internal probability of committing to a tool call, token by token, across the whole 350-token deliberation. My first version, on a linear scale, gave a clean and almost boring answer. Flat. A dead-straight line along the bottom, then a single spike right at the end where it finally commits. Read literally, that says the internal signal never wavers at all, despite all that visible textual agonising. The hesitation is in the words and nowhere else.
This is a trap worth knowing. A linear scale crushes every small value into an indistinguishable zero, and small is precisely where I needed to look. So I replotted on a log scale, which stretches the tiny values out so their shape becomes visible. It's a recurring lesson in this kind of work: when the linear chart looks suspiciously flat, the story is usually hiding below the floor.
The "flat" region wasn't flat. Under the log scale it filled with real, small, repeated spikes running right through the deliberation, and they weren't random: they clustered around the hedging words, the "maybe"s and "Alternatively"s, the exact moments the text was wobbling. So the internal signal is not disconnected from the visible hesitation after all. It tracks it. It's just far, far smaller than anything a linear chart, or any single-token snapshot of the kind I'd relied on so far, could ever have shown.
(Full code: plot_certainty_trajectory in manual_tool_call_v5.py.)
Freezing the decision and looking inside it
The trajectory shows the decision unfolding over time. To see the machinery at the moment of commitment, I froze the first-stage decision, the tool-versus-answer fork, and ran a battery of standard interpretability instruments on it, starting with the logit lens: stopping the model at each internal layer and asking what it would commit to if forced to speak from there, rather than only reading its final answer.
Then two more instruments, both aimed at the same question from different directions: which words was the decision actually leaning on? Attention shows where the model was looking at the moment it chose; gradient attribution, a completely separate calculation, shows which input words the decision was most sensitive to. When two unrelated methods agree, the finding is worth more than either alone.
(Full code: instrument_decision in manual_tool_call_v5.py, which runs the logit lens, attention heatmap, and gradient attribution for each decision stage.)
A second signal nobody had measured yet
At this point I noticed something about every measurement I'd made so far. Each one picked two or three candidate words in advance, GET versus ANSWER, weather versus time, and only ever asked "how sure is the model about these?" Not one of them ever asked the more basic question: how sure is the model in general, about anything, at this moment? Regardless of which words I'd decided in advance to care about?
There's a standard way to measure that, called Shannon entropy. In one plain sentence: it's a single number that sits near zero when the model is confident about what the next word should be, and grows larger the more genuinely-plausible options are competing for the slot. High entropy means "many words would fit here and I'm torn"; low entropy means "obviously this one."
So I built a two-panel chart against one shared timeline. On top, the old signal: certainty about the specific tool-versus-answer choice. Underneath, the new one: the model's general uncertainty about literally the next word, whatever it might be.
The finding was clear and a little humbling. The general uncertainty is high and constantly moving across the entire response, the model is always choosing among several plausible words, the way any writer is always weighing how to phrase the next bit. But it does not track the tool-versus-answer signal above it. The single biggest moment of general uncertainty in the whole response has no matching spike in the tool decision at all. These are two different kinds of not-being-sure, and they move largely independently of each other. Which raised the question that turned out to be the best part of the whole investigation: if the biggest uncertainty wasn't about the tools, what on earth was it about?
(Full code: plot_certainty_trajectory in manual_tool_call_v5.py.)
What was it actually torn between?
A number like "2.4 bits of entropy" means nothing on its own. The only way to make it real is to stop looking at the number and decode the actual words the model was weighing at those peak-uncertainty moments, and read them plainly. So that's what I did: at each highest-uncertainty point, I pulled out the model's real top-five candidate next words and looked at what they were.
The results were not what this project set out to study, and that's the point.
The single highest-uncertainty moment in the first response wasn't about weather or time at all. It was the model deciding whether to start a new paragraph, or carry on the sentence with one of "But," "Let," "Since," or "Wait." Pure sentence-structuring. A writer's choice about rhythm, with no bearing on the actual substance of the answer.
Another peak: a toss-up between "wanted," "wants," and "is." Grammatical tense. Which conjugation to use, not which fact to state.
And the one I keep coming back to. In the second response, after the model already had the real weather data in hand and was simply describing it, the single highest-uncertainty moment was a genuine dead heat between two words: "mild" and "pleasant." Two near-synonyms for a comfortable temperature. Both correct. Both natural. And the model, measurably, was torn between them.
So here is the plain landing point. The model's biggest moments of real, measurable uncertainty across this entire investigation were almost never about the decision the project set out to study. They were about the ordinary things any writer hesitates over: how to open a sentence, which tense to use, which of two perfectly good adjectives to reach for. The "big" decision, which tool to call, was never genuinely in doubt. The small, oddly human-feeling hesitations, "mild" or "pleasant," were where the real uncertainty actually lived.
(Full code: report_high_entropy_moments and the peak-annotation logic in plot_certainty_trajectory, both in manual_tool_call_v5.py.)
Which neuron actually mattered
There's a trap that runs through all of this work: a component which looks like the cause of a decision, by every correlational measure, need not actually be the cause. Something can light up reliably whenever a decision is made without being what drives it. The only way to tell the two apart is to reach in, silence the component, and see whether the decision changes, correlation is a hint, an intervention is proof. I ran that test on this decision's neurons, and found a result worth reporting, though I want to be careful not to over-claim it.
Applying the same silence-and-rerun test here, the specific neuron that turns out to causally matter is, again, not always the one that looks most important. But the more interesting result is a clean asymmetry between the two decision stages. In stage two, the which-tool decision, silencing the top neuron produced a real, sizeable effect: neuron 6610, with a change in the decision gap of Δgap = +5.25, the single largest causal effect measured anywhere in this investigation. In stage one, the tool-versus-answer decision, the same test consistently produced only small or negligible effects across multiple runs. That's not simply "it varies." It's a reproducible split: the second-stage choice has an identifiable, high-impact neuron behind it, while the first-stage choice does not seem to hang on any single unit the same way.
The neuron rankings below show which units look most important for each stage by the usual correlational measure. Worth keeping the trap in mind while reading them: this is the "looks important" view, not the causal one, and the two don't always agree. The rankings are where you start; the silence-and-rerun test is what decides.
I'm flagging this as an open thread rather than a headline finding, because I didn't chase it all the way down, one high-impact neuron in one decision stage is a lead, not a fully mapped circuit. But the asymmetry itself is real and worth stating plainly: silence the right neuron and stage two moves hard; stage one barely notices.
(Full code: analyse_mlp_and_causal_patch in manual_tool_call_v5.py; the evidence is in the printed Δgap numbers, of which +5.25 for neuron 6610 in stage two is the standout.)
Where this goes next
Everything here rests on one real conversation, two turns, and a single deliberately ambiguous prompt. That's enough to find something interesting; it is not enough to call it general. The open question: does this hold up across different tools, different questions, and a model genuinely trying to decide between three or more real options at once, rather than the two-then-two structure the tokenizer forced here?
I'll be honest that the systematic sweep across many prompts was planned and not yet done. It's a real limitation, not a hidden one, and it's the natural next entry. The thing I'll carry forward is the reframing this investigation handed me almost by accident: that the decision we set out to study, which tool to call, was the confident, boring part, and the genuinely uncertain moments were the small human ones we weren't even looking for.
Full runnable code for everything above, and every chart the script produces, is public: github.com/pau1a/Kelstone.