Context Engineering for Operators: Budget, Grounding, and Refusal

A legal-tech firm reduced erroneous AI-generated advice requiring human correction by 90% and cut its core workflow API costs by 62% in six weeks—not by upgrading its LLM, but by implementing a strict context budget that capped input tokens at 40% of the window's capacity. The team had been stuffing up to 40 kilobytes of tangential case files and emails into prompts, assuming more data meant better answers, which led to a 71% human approval rate and spiraling costs. The shift from maximizing context fill to maximizing context relevance treated the window as a constrained production asset, not an infinite scratchpad.

This memo uses a composite scenario from our work with mid-market legal and compliance technology firms. A product team at "LexiPro," a provider of AI-powered paralegal assistance tools, faced a critical performance bottleneck. Their flagship feature, which summarized case law and cross-referenced it with internal corporate policies, was failing in subtle but dangerous ways.