Making a Frontier Model ~52% Cheaper and ~74% Faster — Without Downgrading It

The application generates long, structured technical documents — multi-section proposals with scope classification, costing, schedules, and rendered artifacts — from short free-text briefs. Generation ran on a premium frontier reasoning model (Claude Opus 4.8), and the output quality was exactly what the workflow needed. The model was not the problem. How we were calling it was.

Every proposal ran as one long, serial generation: a single call that produced the entire document end to end. That had two costs. Latency: a full run took on the order of ~160 seconds — slow enough to hurt the interactive experience and to bottleneck any batch use. Cost: at frontier token rates, an output-heavy proposal ran to roughly $0.87 in model spend, and because everything was generated in one monolithic pass, there was no way to bound or shape where that spend went.

The usual reflex here is to reach for a cheaper model. We deliberately did not. The question we tested was not “do we need Opus 4.8?” — the quality bar said we did — but “are we calling Opus 4.8 efficiently?” The answer was no.

A long proposal is not one indivisible thing. It is a set of largely independent section groups — narrative, technical, and review content — that do not depend on each other to be written. The serial pipeline was generating them one after another anyway, paying for that ordering in wall-clock time.

• Parallel section generation. We restructured the pipeline to generate the independent section groups concurrently on the same Opus 4.8 model, bounded by a configurable concurrency limit (an async semaphore), with a per-section timeout and partial-failure salvage so one slow or failed section never sinks the whole document. End-to-end time collapses from the sum of every section to roughly the slowest one.
• Tighter, scoped calls instead of one monolith. Each concurrent call generates only its own section group against a shared, pre-computed context, rather than one giant call carrying the entire document in a single prompt/response. Shorter, focused generations — combined with prompt caching on the shared context across the concurrent calls — are what drove the per-proposal cost down, on the same model.
• Compute-once upstream stages. Deterministic upstream work (scope classification, scaffolding, costing) is computed once and passed forward as shared context rather than re-derived, keeping repeated work out of the model’s billing path and making the shared prefix cacheable across the concurrent section calls.
• No model downgrade. Every number below is on Claude Opus 4.8 before and after — the frontier model is held constant. The savings are entirely from how the model is invoked, not from trading quality for price.

Measured on production runs of the same Opus 4.8 model, before vs after the parallel-section restructure. Same model, same quality bar — only the call structure changed.

Because the model never changed, the risk was not “is the cheaper model good enough” — it was “did splitting one call into concurrent section groups degrade coherence or coverage?” Every cost and latency win only counted if it survived two independent checks: a deterministic evaluator (structure, required sections, scope coverage) and a semantic parity judge (an independent LLM scoring the restructured output against the original serial-Opus output). The parallel output held the bar — verified by an independent judge, not by the team that built it.

• Serial generation was a self-imposed tax. Writing independent section groups one after another made total time the sum of every call and forced one monolithic, expensive prompt/response. Concurrency made latency the slowest call; scoping made each call cheaper — same model, same output quality.
• The instinct to downgrade the model is often the wrong lever. The most defensible win here was not swapping Opus 4.8 out — it was calling it efficiently. That keeps frontier quality and still roughly halves the cost.
• Bounded concurrency with salvage, not naive fan-out. A semaphore caps parallelism, a per-section timeout stops a single slow call from stalling the document, and partial-failure salvage keeps a completed document even if one section needs a retry.
• Compute deterministic work once. Upstream stages that don’t need the model (scope classification, scaffolding, costing) are derived once and passed as shared, cacheable context — not recomputed or re-sent per call.
• Trust requires independent verification. Cost and latency wins are only real if an independent quality gate confirms the restructure didn’t degrade output — so the savings are defensible to a customer, not just to ourselves.

For any LLM-powered application generating long, multi-part output on a frontier model — before you consider downgrading the model:

• Check the call structure first. If your output has independent sections, one long serial generation is likely costing you both latency and money that a downgrade would only partly recover.
• Parallelize independent section groups on the same model, bounded by a semaphore, with per-section timeouts and partial-failure salvage. Latency goes from sum-of-calls to slowest-call.
• Scope each call and share a pre-computed context so prompt caching applies across the concurrent calls — this is where the cost drop comes from without touching the model.
• Compute deterministic / repeated upstream stages once and pass them as shared context, out of the model’s billing path.
• Freeze the original frontier output as the quality reference and gate every change behind an independent evaluator, so parity is provable.
• Only after that, if the numbers still demand it, evaluate a right-sized model — as a second, separately-verified step, not the first reflex.