Local models deserve a first-class harness

Local models are often offered as the privacy checkbox beside the real product. They get a text box, the same enormous prompt as a frontier model and an apology when the result collapses.

That is not first-class support. It is equal neglect.

A useful harness understands capability. Can the model see images? Does it reliably call tools? How much context can it use before quality degrades? Which prompt shape fits its native protocol? Those facts should control the work a model receives and the tools exposed to it.

The honest half matters just as much. Running locally does not make a model qualified for every task. A small model may be excellent for classification, extraction and private drafting while being the wrong choice for a difficult repository-wide change. If an allowed hosted model is materially better, Miton should say so and explain the cost and privacy boundary. If hosted inference is not allowed, it should work within that constraint without pretending the constraint vanished.

First-class means participating in the same routing system, cost story and project context as hosted models. It means receiving compact, relevant context rather than the leftovers of a cloud-optimised prompt. It means local providers are tested as real execution paths. It also means a failure can be described accurately: missing capability, exhausted context, disconnected process—not “AI error”.

Local inference will keep improving. The right response is not to wait for it to imitate the cloud. Build the harness that can recognise what each model is already good at, protect the work it keeps private and route around its honest limits.