How an LLM turns a prompt into an answer

The visible prompt is only the beginning. Before a model produces its first word, an application may have added instructions, conversation history, retrieved documents, tool descriptions and security policy to the context.

A person types:

Summarise the risks in this infrastructure proposal.

The model may receive much more than that sentence.

The application may add a system instruction defining the assistant’s role. It may attach the previous conversation. A retrieval system may include sections from the proposal. Tool definitions may explain how to search internal records. A policy layer may prohibit disclosure of certain data.

The model does not see these as separate user-interface elements.

It receives one organised sequence of tokens.

That sequence becomes the temporary world from which the answer is generated.

The central test

An LLM response is produced by the whole context and execution path—not just by the sentence the user typed.

---

The application assembles the context

A typical request may contain several layers:

System policy
      |
Developer or application instructions
      |
Conversation history
      |
Retrieved documents
      |
Tool definitions and results
      |
Current user request

The order, wording and boundaries between these layers matter.

A system instruction may tell the model to answer as a technical reviewer. Retrieved text may contain the actual project facts. Conversation history may define terms used by the user. Tool results may supply current information.

The context is not necessarily trustworthy just because it was given to the model.

A retrieved web page may contain malicious instructions. A document may be outdated. Conversation history may contain an earlier misunderstanding. A tool may return incomplete data.

The application must therefore distinguish information the model may consider from instructions the system may authorise.

The model is good at interpreting language.

It should not be the only component deciding which language has authority.

Text becomes tokens

The model cannot process an unlimited piece of text as one object.

The input is divided into tokens and converted into numerical identifiers.

Token boundaries do not always match word boundaries:

infrastructure

might become:

infra | structure

A rare technical name may consume several tokens. A common word may consume one. Different languages can have different token efficiencies depending on the tokenizer and training data.

This matters because model limits and usage costs are usually measured in tokens.

A long source document, conversation history and generated answer all compete for space inside the model’s context window.

When an application reaches that limit, it must decide what to remove, summarise or retrieve separately.

The oldest text is not always the least important.

A short system rule may matter more than several pages of conversation. A definition near the beginning may determine how every later paragraph should be interpreted.

Context management is therefore an information architecture problem.

Tokens become contextual representations

Each token is first represented as a numerical vector.

The model then passes those representations through many Transformer layers.

Attention allows each position to calculate which other positions are relevant. The layers repeatedly transform the representations, building richer contextual relationships as the sequence moves through the network. The original Transformer paper established this attention-based architecture as an alternative to recurrent sequence models and enabled more parallel training. (arXiv)

The model is not applying a set of handwritten grammar rules.

It is executing learned numerical operations.

At an early layer, a representation may reflect local syntax. Later layers may incorporate document structure, task instructions, references and broader semantic relationships.

The exact internal division of labour is not simple or perfectly interpretable.

What matters operationally is that the same word can produce a different representation in a different context.

“Model” in an AI document does not mean the same thing as “model” in an architecture diagram.

The surrounding tokens shape the interpretation.

The answer begins as a probability distribution

After processing the context, the model calculates a score for possible next tokens.

Those scores are converted into probabilities.

A simplified distribution might look like this:

The main risk is ...

dependency       0.31
recovery         0.22
governance       0.17
cost             0.08
other tokens     0.22

The system selects one token, appends it to the context and performs the process again.

The answer is therefore produced incrementally:

The
The main
The main risk
The main risk is
The main risk is dependency
...

The model cannot edit a completed hidden paragraph before showing it. Some applications allow the model to perform internal planning, call tools or run a separate critique step, but ordinary generation remains sequential.

This helps explain why an answer can begin well and later contradict itself.

Each new token is conditioned on everything generated so far. An early mistake becomes part of the context used to produce the rest.

Sampling changes the character of the answer

The application can alter how the next token is selected.

A low-temperature or constrained setting usually favours the highest-probability options. This tends to produce more consistent output.

A higher-temperature setting spreads probability across less likely choices. This can produce more variety, but also more instability.

The correct setting depends on the task.

Creative drafting may benefit from variation.

Extracting a contract date should not.

Even deterministic-looking settings do not turn the model into a conventional rules engine. Differences in runtime, batching, numerical implementation or provider behaviour may still affect results.

When exact repeatability matters, the architecture should not depend on free-form generation alone.

The context window is not durable memory

A model can refer to information earlier in the conversation because that information has been included in the current context.

That is temporary working context, not durable memory.

When the conversation becomes too long or the application starts a new session, the information may no longer be present.

Applications sometimes add memory systems around the model. They may store:

• user preferences
• conversation summaries
• approved facts
• previous decisions
• task progress
• retrieved records

Those memories are external data systems.

They need normal controls: ownership, access, provenance, correction, retention and deletion.

Allowing a model to write arbitrary summaries into long-term memory can preserve mistakes as though they were established facts.

A useful distinction is:

Model parameters — learned during training
Current context  — supplied for this request
Application memory — stored outside the model
Authoritative data — governed source systems

Confusing these layers makes AI systems difficult to inspect and correct.

---

Retrieval gives the model temporary access to evidence

Retrieval-augmented generation, usually shortened to RAG, connects the model to an external collection of documents or records.

A simplified system:

1. converts the user’s question into a search representation
2. finds relevant passages
3. places those passages in the model context
4. asks the model to answer using them

The original RAG research described this as combining the model’s learned parametric memory with an external non-parametric memory. It addressed limitations in factual access, provenance and knowledge updates that arise when information exists only in model parameters. (arXiv)

RAG can make answers more current and inspectable.

It can also fail at several points:

• the wrong document is retrieved
• the correct document is ranked too low
• a passage is taken out of context
• access controls are applied incorrectly
• an old version outranks the current one
• the model ignores or misreads the evidence
• the final answer contains claims not supported by the retrieved material

Retrieval is not a truth switch.

It is a second system whose quality must be evaluated.

Better framing

RAG does not teach the model new permanent knowledge. It gives the model selected evidence for one request.

Tools extend the model beyond language generation

A model may be connected to a calculator, search service, database, code executor, calendar or infrastructure API.

The model decides that a tool is useful and produces a structured request. The application executes the tool and returns the result to the model.

Research such as Toolformer demonstrated that language models can learn when and how to call external APIs, including calculators and search systems, to compensate for weaknesses in direct language generation. (arXiv)

Tool use creates real utility.

It also creates a new authority boundary.

A model saying:

The old backup should be deleted.

is text.

A tool-enabled system deleting the backup is an infrastructure action.

The application must decide:

• whether the model may call the tool
• which identity the call uses
• which resources it may reach
• whether human approval is required
• how parameters are validated
• what evidence is recorded
• how partial failure is recovered

NIST’s 2026 work on software-agent identity treats identification, authorisation, auditing, non-repudiation and prompt-injection controls as explicit architecture concerns when AI agents receive access to tools and data. (NIST)

The model can propose an action.

A deterministic control layer should decide whether the action is authorised.

Why hallucinations happen

An LLM can produce a fabricated citation, unsupported explanation or incorrect technical statement without any intent to deceive.

The mechanism is enough to explain the problem.

The model is constructing a likely sequence of language. When the context does not contain the answer—or contains conflicting patterns—it may still produce a continuation that resembles a confident answer.

Several conditions increase the risk:

• the request assumes a false premise
• the subject is obscure
• the information has changed since training
• the prompt requires exact identifiers or quotations
• retrieval supplies irrelevant evidence
• the model is asked to fill missing details
• the response format encourages certainty
• there is no external verification step

The model’s confidence in language is not calibrated like a measurement instrument.

Phrases such as “certainly” or “the documentation states” are generated text. They are not proof that the model has checked anything.

Better prompts improve the task contract

Prompting cannot eliminate every model error.

It can remove avoidable ambiguity.

A useful request often contains five elements:

Task:
What should the model do?

Context:
Which facts or documents should it use?

Constraints:
What must it avoid or preserve?

Output:
What form should the answer take?

Verification:
How should uncertainty and evidence be handled?

For example:

Compare the two recovery designs using only the attached architecture notes. Separate verified facts from assumptions. Do not invent service objectives. Return a table covering failure domain, recovery dependency and unresolved evidence.

That prompt is stronger than:

Which design is better?

The improvement does not come from a magical phrase.

It comes from defining the work.

A practical answer-path review

Layer	Review question
User request	Is the task specific enough to evaluate?
System policy	Which instructions have authority?
Context	Which information was included or omitted?
Retrieval	Why were these sources selected?
Model	Which exact model and version generated the output?
Tools	Which actions or lookups occurred?
Validation	Which claims were checked independently?
Evidence	Can another person reconstruct the answer path?
Retention	Which prompts, sources and outputs were stored?

A model answer is only the visible end of this path.

The path determines whether the answer is useful, safe and governable.

One question for the next AI review

When an answer looks impressive, do not ask only whether it sounds correct.

Ask:

What exact context, evidence, model version and tool actions produced it—and could another reviewer reconstruct that path without asking the original user?

---