The interesting question about language models is no longer what they can do. It is which problems they should be pointed at, given that a great many tasks now routed through them were solved decades ago by methods that are faster, cheaper, and produce the same answer every time.
There is a reasonable first test. If the rule can be written down completely, write it down. A model is for the cases where it cannot be.
Where a deterministic solution wins outright
Anything with an exact definition of correctness belongs in code. Validating a format, performing arithmetic, transforming between two known schemas, sorting by explicit criteria, matching against a fixed set. These have a correct answer that can be computed, and computing it takes microseconds rather than seconds.
The temptation is understandable, because the model handles messy input gracefully and requires no schema definition up front. That flexibility is genuinely convenient during development, and it is being purchased with latency, cost, and the permanent loss of any guarantee about the result.
The clearest sign of a misapplication is finding yourself writing a prompt that specifies the rules exhaustively. At that point the rules exist, they have simply been written in prose and handed to something that will follow them approximately.
Where classical methods still win
Between rules and language models sits a large body of ordinary machine learning that is frequently skipped over.
For high volume classification with labelled examples, a purpose trained classifier is typically faster by orders of magnitude, cheaper by more, and produces a calibrated confidence score that can be thresholded. For prediction over tabular data, gradient boosted trees remain the stronger approach and are considerably easier to explain. For anomaly detection over metrics, statistical methods do the job without any model at all.
These approaches share properties that matter in production. They are deterministic, they are inspectable, their behaviour changes only when you change them, and their failure modes are understood.
Latency, cost, and volume
The gaps here are large enough to change what is buildable rather than merely what is efficient.
A regular expression or a lookup returns in microseconds. A small trained classifier returns in single digit milliseconds. A model call takes hundreds of milliseconds to seconds, and cannot be made much faster, because generation is inherently sequential.
At low volume that difference is invisible. At high volume it determines architecture, because a per item cost that is negligible at a thousand items a day is the dominant line item at ten million. Any task in a hot path, or applied to every record in a large set, deserves an explicit justification for why a cheaper method will not do.
Determinism, audit, and reproducibility
Some requirements are simply incompatible with a probabilistic component, and it is better to recognise that early.
If a decision must be explained afterwards in terms of the rule applied, a model cannot supply that, because its stated reasoning is generated text rather than a record of how the output was produced. If the same input must yield the same output, a model does not offer it. If an auditor needs to verify that a policy was applied consistently across a year of decisions, the only defensible implementation is one where the policy exists as code.
This is not an argument that models have no place in regulated work. It is an argument that they belong in the parts where judgement was always required, and not in the parts where the answer was determined by a rule.
The maintenance cost nobody budgets
A rule based system behaves identically until someone changes it, which is an underappreciated property. A system depending on a hosted model does not have it.
Providers update models, deprecate versions, and adjust behaviour, and a prompt tuned against one version can behave differently against its successor in ways that are subtle rather than obvious. Nothing failed, the output simply shifted, and unless there is an evaluation set running continuously the shift is discovered by a user rather than by a test.
That implies an ongoing obligation. Pin versions where the provider allows it, maintain a representative task set with a mechanical definition of success, and run it on every model change rather than only on your own changes. That work is real and recurring, and it should be weighed at the point of choosing a model rather than discovered afterwards.
The three options compared
| Approach | Best for | Latency | Deterministic |
|---|---|---|---|
| Rules in code | Exact, definable answers | Microseconds | Yes |
| Trained classifier | High volume with labelled data | Milliseconds | Yes |
| Language model | Unstructured input, long tail | Hundreds of ms to seconds | No |
The middle row is the one most often skipped, and it is frequently the correct answer for the tasks a model is being asked to do at volume.
What models are genuinely for
Stating the positive case sharpens the negative one.
They are strong where the input is unstructured language and the desired output is language. They handle a long tail of variation that would require an unbounded number of rules. They work without training data, which makes them viable where labelling was previously the obstacle. And they tolerate ambiguity, producing something reasonable where a rule based system would simply reject the input.
Those properties are the reason to use one, and the honest test is whether the task actually has them.
The pattern that works
Most real systems are better as a hybrid, with a deterministic core and a model at the edges.
Use the model to interpret messy input into a structured form, then act on that structure with ordinary code. Use it to handle the cases a rule based path could not classify, rather than for every case. Validate what it produces against a schema, so that a malformed result fails immediately rather than propagating.
This keeps the guarantees where guarantees are needed and applies the flexibility only where it is genuinely required, which is usually a much smaller portion of the system than the initial design assumed.
Note: a useful check before adding a model to a path is to ask what happens on the day it returns something confidently wrong. If the answer is that nothing downstream would notice, the problem is the architecture rather than the model.