Term
Path Dependency (LLM Output)
Path dependency describes how, during LLM generation, each early token shapes everything that follows — the first verbalized finding steers the entire remaining analysis in one direction.
Path Dependency — explained in detail
A language model produces text autoregressively: token by token, each new token conditioned on all the ones before it. Path dependency arises directly from this. As soon as an early step — partly determined by random sampling — heads in a particular direction, that beginning becomes part of the context for everything that comes next. The model does not continue “neutrally from a bird’s-eye view”; it consistently builds on what it has already written itself.
In concrete terms: the first finding the model verbalizes acts like a switch on a railway track. Once it has started describing a particular problem, its own prior output increases the probability of staying in the same lane. Other possible findings that a different opening would have surfaced fall out of focus. The output is therefore not only a function of the input, but also of the model’s own early generation path — the “path” that generation happened to take by chance.
Path dependency is the downstream consequence of sampling randomness, not the randomness itself. The randomness decides which first step is chosen; path dependency ensures that this single step has a disproportionate effect on the overall result.
Example / Practical use
Have the same model analyze the same code twice and it will often find different things. At first this seems contradictory — the code is identical, the model is identical. The explanation is path dependency: on the first run the model might mention a performance issue first and work outward from there; on the second run the first hit lands on a security flaw, and the entire subsequent analysis organizes itself around that entry point.
In practice this means the order of findings shifts the overall result. To get more reliable or more complete analyses, run the model several times and merge the results rather than relying on a single pass — or structure the prompt so the model gathers broadly first before committing to any one finding.
Distinction from related terms
Pure sampling randomness (temperature, top-p) only describes that token selection is probabilistic — that different tokens can be chosen at all. Path dependency describes the consequence: that a single early random decision disproportionately shapes the rest of the run. Randomness is the cause; path dependency is the amplifying mechanism.
Hallucination is also something different: there the model produces something factually false. With path dependency, each individual output can be correct — it is the selection and order that vary between two runs, not the truth of the individual statement.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarReasoning Effort
Reasoning Effort is a control parameter on modern reasoning models that sets how much internal step-by-step thinking (thinking tokens) a model spends before answering — higher levels raise quality but also latency and cost.
LexikonLLM Hallucinations — Causes and Remedies
Why LLMs confidently invent falsehoods, what types of hallucinations exist, and which remedies actually help — RAG, source enforcement, verification.