Prompt engineering has become the default answer to almost every AI performance or cost problem I get asked about, and for most of the cases I've looked at closely, it's the wrong place to start. A badly specified prompt genuinely can sink a system, so this isn't an argument that prompts don't matter. It's an argument about order. By the time a team is three or four rewrites deep into a system prompt, chasing another few points of accuracy or another few cents off the per-call cost, they have usually skipped three cheaper questions that would have moved the number further: whether the workload needs the model they're running it on at all, whether it needs a model that size, and whether the calls around it are structured sensibly in the first place. Tuning the prompt is the move that gets made first because it's the one move nobody else has to sign off on. It should be the one made last.
Take a version of this. A team running a support-ticket triage pipeline — classify the ticket, extract the account details, draft a suggested reply — had spent five months on the system prompt for the classification step, alone. It started at roughly six hundred tokens telling the model what the categories were. By the fifth month it was past three thousand: edge-case instructions for tickets that mentioned two products, a paragraph on how to handle sarcasm, a list of eleven phrases the model kept over-triggering on, appended one at a time as each new failure mode showed up in a weekly review. Accuracy had moved from 81% to 87% over that period, at the cost of a prompt nobody but its author could safely edit anymore. Nobody on the team had tried running the same classification task on an open-source model self-hosted on infrastructure they already paid for, and nobody had tried a smaller model in the same family as the one they were using — either test would have taken an afternoon. Nobody had asked why classification, extraction and drafting ran as three separate calls that each re-read the full ticket text, when one well-structured call could have done all three off a single read. When someone finally tried that, changing nothing about the prompt itself, accuracy on the classification step went to 91% and the per-ticket cost dropped by half, because the model doing the classifying now had the extracted account context to reason from and wasn't repeating the same read three times.
The order that would have found that answer sooner isn't complicated, and it doesn't require anyone's permission for the first two steps. Try the workload on an open-source model first, because for a large share of tasks — classification, extraction, drafting from a template — the frontier model was never necessary, and the test costs an afternoon. If that doesn't clear the bar, try a smaller model in the same family before touching anything else; a task that genuinely needs frontier-class reasoning is rarer than the volume of frontier-class API calls being made would suggest. Only after both of those are ruled out does architecture become the question worth a week: how many calls this actually takes, what each one is reading, and whether anything is being asked twice. The prompt is the last of the four. It still matters, but by the time the first three questions get answered honestly, most of what the prompt was being asked to compensate for has usually already been fixed somewhere cheaper.
The reason prompt engineering wins by default has nothing to do with which lever produces the bigger number. It's which lever one person can pull alone, from their own editor, without a procurement conversation, an infrastructure ticket, or an admission that a model choice made eight months ago should be revisited. Swapping to an open-source model touches a vendor relationship and maybe a compliance review. Dropping to a smaller model means telling whoever approved the current one that the extra capability wasn't needed. Restructuring the calls means redesigning something already in production and owned by someone else's sprint. Editing a system prompt touches none of that, and it's a task with no natural end point — there's always one more edge case to add a clause for — so it produces a steady stream of small, visible wins that look like progress in a weekly review, right up until the twentieth edge case makes the twenty-first one worse in a way nobody can trace anymore.
The prompt that grows this way doesn't just cost the hours spent writing it. It becomes the least durable part of the system, because it's tuned against the specific quirks of one model version, and the vendor's next release routinely breaks half the carefully worded workarounds without warning. A smaller model swapped in cleanly is a config change. A restructured call graph is a design that survives the next model upgrade largely intact. A three-thousand-token prompt held together by eleven accumulated edge cases has to be partially rewritten every time the model underneath it changes, by the one person who still remembers why half of it is there.
The test worth running before the next round of prompt rewrites is cheap enough to finish in a day: has anyone actually tried this exact workload on a smaller or open-source model this month, and can they say with data why it fell short — or is the frontier model choice just an assumption nobody has re-checked since the pipeline was first built. If the honest answer is the second one, the prompt sitting on someone's desk for another rewrite is the most expensive fix on the table, and it's the one being reached for first anyway.
That's the ninth pattern.