Artificial Intelligence, December 2024

When Not to Reach for an LLM

A reliable sign that someone is new to these tools is that they apply a language model to everything. The reflex is understandable, the model is impressive and it will attempt any task presented to it, but the more valuable and harder won skill is recognizing when a model is the wrong instrument, because a substantial fraction of the time it is, and the alternative is not merely cheaper, it is correct by construction in a way a probabilistic system cannot match.

A language model is the wrong tool when the task is deterministic and well specified. Parsing a structured file, validating a format, performing arithmetic, transforming data from one schema to another, these have exact and knowable answers, and a few lines of conventional code will produce the right one every time. A model will usually produce the right one as well, and usually is precisely the problem, because the occasional confident error is the one that passes review and reaches production, whereas the deterministic implementation does not have that failure mode at all.

The same logic applies wherever correctness must be guaranteed or reproducibility is required. Regulatory logic, financial calculation, and anything auditable should be expressed as code that can be read, tested, and relied upon to behave identically on identical input, not delegated to a system whose output can shift with phrasing or model version. A model is likewise the wrong choice wherever a simpler, established method already suffices, a database query, a regular expression, a lookup table, each of which is faster, cheaper, and incapable of inventing an answer. Where a model genuinely earns its cost is the messy middle, the tasks that resist exact rules, interpreting intent in unstructured text, summarizing a long document, drafting a first version, converting loose natural language into structure. These are the problems conventional code handles poorly, and they are exactly where the flexibility of a model is worth its unpredictability.

A language model is a genuinely new capability, but it is a specific one, not a universal solvent for every problem that involves text or data. The engineers who use it to greatest effect are not those who use it most, they are those who have developed a clear sense of where its strengths apply and where a deterministic tool is simply the better answer, and who reach for the model deliberately rather than by reflex. That judgment, knowing which problems belong to which class of tool, is becoming one of the more valuable competencies in building software well, precisely because the temptation to apply an impressive new instrument to everything is so strong and so rarely questioned.