There is a point at which further effort on the wording of a prompt stops paying. It arrives sooner than most people expect, and once past it the remaining quality is not in the instruction at all. It is in what surrounds the call.
A single prompt shapes one forward pass. A working system calls the model repeatedly, feeds it the results of its own previous actions, and decides when to stop. That structure, not the wording, is where the difficulty lives.
What prompt engineering actually buys
Careful wording is genuinely useful, and it is worth being specific about why. Clear instructions reduce ambiguity about the task. Explicit output formats make the response parseable. Worked examples establish a pattern the model can follow. Constraints stated as requirements rather than suggestions are more likely to be respected.
All of that is real, and all of it is bounded. Once the instruction is unambiguous, the format is specified, and the examples are representative, further rewording produces variation rather than improvement. Time spent there is time not spent on the parts of the system that are still failing.
The loop is the program
An agentic system is a control loop. The model receives a state, proposes an action, the runtime executes it, and the result is fed back for the next decision. That cycle repeats until something ends it.
Everything interesting is in the runtime rather than the model. What state is assembled for each call. Which actions are permitted. What happens when an action fails. How many iterations are allowed before the loop is abandoned. The model contributes judgement at each step, and the surrounding code determines whether that judgement is usable.
This reframes the debugging question. When an agent fails, the useful question is rarely whether the prompt was worded correctly. It is which step produced the first wrong decision, what information that step actually had, and why the loop did not detect the problem.
Context is a budget, not a container
The most common structural mistake is treating the context window as somewhere to accumulate history. Append every turn, every tool result, and every error, and the system will work for a while and then degrade in ways that look like the model getting worse.
Two things go wrong. Cost and latency rise with every call, because the entire accumulated context is processed each time, and a long running loop pays that repeatedly. And retrieval within a long context is uneven, with material in the middle of a large context attended to less reliably than material at either end.
The alternative is to treat context as a budget allocated deliberately on every call. Keep durable state in ordinary storage outside the model. Summarise completed work rather than carrying its full transcript. Retrieve only the history relevant to the current step. Reserve a known share of the budget for the actual task rather than letting accumulated history consume it.
Tools change the failure modes
Once the model can call tools, the failure surface changes character. A wrong sentence is a quality problem. A wrong action is an operational one.
The practical defences are ordinary engineering. Validate arguments against a schema before executing anything, because a plausible looking call can still carry a malformed or out of range value. Make actions idempotent where possible, since retries will happen. Return errors as structured, readable results rather than raw stack traces, because the model has to reason about them. And keep tool descriptions precise, since the model chooses between tools based largely on how they are described.
There is a subtler consequence. Anything returned by a tool enters the context as text, and the model does not reliably distinguish content it retrieved from instructions it was given. Content fetched from an external source is untrusted input, and it is being fed directly into the thing making decisions.
Structure beats instruction
A recurring lesson is that constraints enforced by the runtime outperform constraints requested in the prompt. Asking for output in a particular shape works most of the time. Requiring it, by validating against a schema and rejecting anything that does not conform, works every time, because the failure becomes visible immediately rather than propagating into the next step as malformed state.
The same principle applies more broadly. If a step must not exceed a spend, cap it in code rather than instructing the model to be economical. If an action is irreversible, gate it behind an explicit confirmation rather than a warning in the system prompt. Anything expressed only as an instruction is a preference the model will usually respect. Anything expressed in the runtime is a property the system has.
Recovery is the hard part
Loops fail in characteristic ways, and none of them are fixed by better wording.
A model that receives an error will often retry the same action unchanged, because nothing in its state indicates that repetition is futile. It will sometimes declare success without having achieved anything, particularly when the task was underspecified. And it will occasionally alternate between two approaches indefinitely, each looking reasonable in isolation.
The defences are budgets and checks. Cap the number of steps. Cap total spend. Detect repeated identical actions and interrupt rather than allowing them. Verify completion against an external condition rather than accepting the model's own assertion that it finished. A loop with no termination guarantee is not a system, it is an expense.
Evaluating something that is not deterministic
The same input does not produce the same output, which breaks the usual testing approach. Asserting on exact output produces tests that fail for no reason, and relaxing the assertion until it passes produces tests that catch nothing.
What works is measuring outcomes across a set of representative tasks and tracking the success rate, rather than checking any individual response. That requires a fixed task set with a definition of success that can be checked mechanically, and it requires accepting a rate rather than a guarantee.
It also requires traces. When a run fails, the useful artefact is the full sequence of states, decisions, and results, because the failure is almost always several steps upstream of where it became visible. Systems built without that record are debugged by guesswork.
Note: the useful test of a design is whether it degrades or collapses. A loop that runs longer, costs more, and returns a worse answer is recoverable. A loop that cannot terminate, or that reports success it did not achieve, is not.