Wiring a Local LLM Into Real Work — OpenAI-Compatible APIs and MCP

The model is running on your machine. You type a question into the chat window and something plausible comes back. Most people stop here.

The value is on the other side of that. Read a file. Do a calculation. Search the documents your team actually keeps. A model becomes a tool only once it can reach outside itself. Tool calling is the bridge, and MCP is what standardises the far end of it. This article covers the structure of connecting a local model to real work, using only what the official documentation states — and saying so where it does not.

It works in the chat window, and stops right after

The reason is simple: a chat window is an entrance, not a workplace. Real tasks need outside information. Today stock count. The dimensions on this drawing. Last week measurement log. The model knows none of it.

The fix has two halves. First, make the model callable from outside. Second, make the model able to call outside. The first is an OpenAI-compatible API; the second is tool calling. Two passages running in opposite directions, and you need both open.

OpenAI compatibility as the shared language

Every major runtime exposes an OpenAI-compatible endpoint. That is the first passage.

The llama.cpp llama-server documents OpenAI API compatible chat completions, responses, and embeddings routes, and ships /v1/models, /v1/completions, /v1/chat/completions, /v1/embeddings and /v1/responses. LM Studio offers the same five, defaults to port 1234, and states that pointing an existing OpenAI client at http://localhost:1234/v1 is all that is required. vLLM adds the Anthropic Messages API and gRPC on top of OpenAI compatibility.

Ollama deserves a more careful reading. Its documentation uses the deliberately bounded phrase compatibility with parts of the OpenAI API, and then lists what is not supported.

Reading that list as a weakness is the wrong instinct. Publishing the boundary is itself useful in practice. Whether the other runtimes cover the same surface is not something their documentation establishes. What is established is where Ollama stops.

One version note: /v1/responses was added in Ollama v0.13.3. When you read older write-ups, watch for this kind of addition date.

What tool calling actually does

The second passage is tool calling. The Ollama documentation defines it as letting models call tools and incorporate the results into their responses.

1. You hand over a list of available tools, each with a specification.
2. The model replies that it wants this tool called with these arguments.
3. Your program is what actually calls it.
4. You return the result to the model.
5. The model answers with that in hand.

Step three is the one people misread. The model does not execute anything. It emits a structured statement of intent. Everything about what runs, with what permissions, and whether it should run at all stays on your side of the line — which is also where the security review belongs.

Three calling patterns

The documentation shows three shapes.

Single calls one tool and includes the result in the next request. Simplest, and enough for a lot of real work.

Parallel requests several tool calls at once and returns the results together. Useful when the pieces of information are independent.

Multi-turn lets the model call tools across a conversation whenever it decides it needs to. This is the agent loop. Flexible, and it needs an explicit stopping design.

A rule of thumb: if you already know what information is needed, single or parallel covers it. If the next action depends on the previous result — check stock for this part, and if it is short, look at the order history — you are in multi-turn. The inverse matters more: building multi-turn where no dependency exists just hands the model extra decisions to get wrong.

Starting local, start with single. Get that stable, widen to parallel, and reach for multi-turn only where nothing else fits. Build multi-turn from the start and a failure tells you nothing about which stage caused it.

One caution when combining tool calls with streaming: the documentation instructs you to collect all thinking, content and tool_calls chunks and return them together with the tool results in the next request. Take part of it and return part of it and the state breaks.

How far a local model actually gets

This is the part readers most want a number for, so let me be straight: the official documentation does not publish a list of models that support tool calling. The current Ollama docs use qwen3 in every code example and name nothing else.

Lists of supported models circulate. I could not corroborate them against current official documentation, so I am not reproducing them here. The procedure instead: check the model card and the tag notes yourself for tool-calling support. That is the reliable route.

There is likewise nothing official about limitations specific to small models. Absence of documentation is not evidence of absence, so the answer is to try it on your own work.

When it fails, it fails in one of two ways. Either the argument types come apart — a string where a number belongs, a missing required field — or the model calls a tool that was never needed. Both are cheap to catch if you validate arguments against the schema before executing anything, which you should be doing regardless.

MCP — standardising the tool side

Tool calling solves one application talking to one model. MCP solves the other axis: write the tool once and every compatible application can use it.

Three parties. The Host is the AI application itself, creating one Client per server it connects to. The Server is the program offering tools and data, running locally or remotely.

Communication splits into two layers. The data layer is built on JSON-RPC 2.0. The transport layer comes in two forms: Stdio for inter-process communication on the same machine, with no network overhead, and Streamable HTTP, which combines HTTP POST with optional Server-Sent Events, for remote servers. Keeping everything local means Stdio.

That split has a practical payoff. The same JSON-RPC 2.0 message format rides on either transport, so you can move a tool between local and remote without rewriting it. Build it on Stdio first and expose it later, or the other way round. The cost of moving is low.

Security follows the transport. Streamable HTTP supports ordinary HTTP authentication such as bearer tokens and API keys, and the specification recommends OAuth for obtaining them. Stdio is process-to-process on one machine, so network authentication never arises. The all-local configuration is simpler here too.

The three server primitives

Each is enumerated with a list method and tools are executed with tools/call. The design assumes the available set can change at runtime.

The separation is deliberate. Tools are execution with side effects, Resources are read-only information, Prompts are conversational shape. Things with different properties do not share a slot. When you build your own server, sorting your capabilities into those three buckets is usually where the design becomes clear.

What the version change deprecated

This is the part that matters when reading older articles. Protocol version 2026-07-28 changed the client-side primitives.

Elicitation is current: the server asks the user for additional input. Useful ahead of an operation you want confirmed.

Sampling is deprecated. It let a server request a completion from the client language model; new implementations are directed to integrate with an LLM provider API directly. Logging was deprecated in the same version, with stderr or OpenTelemetry recommended instead.

A great many MCP explainers still present both as current features. Start designing without checking the version and you will build on something already deprecated. Reading the specification version number is not optional in this area.

The Sampling deprecation also reads as a shift in intent. Letting a server borrow the client model was a way to keep servers model-agnostic. Withdrawing it in favour of direct integration narrows the protocol toward passing context and nothing else — and the documentation says as much, noting that MCP focuses on the protocol rather than on how an AI application uses models. A clearer division of labour makes your own side easier to reason about.

What it is good at, and what it struggles with

The split is sharp. What goes through reliably is work with a fixed input and output shape: classification, format conversion, summarisation, extraction, code completion. Tool calls are most stable when the arguments are few and simply typed.

What struggles is long multi-step reasoning and strict structured output repeated many times. A chain of conditions ending in a conclusion tends to lose the thread partway. Matching a complex schema exactly, dozens of times in a row, breaks somewhere.

The test to apply: is failure cheap on this job? Cheap failure suits local. Needing it right first time suits the cloud. That line sits directly on top of the cost argument in reading your cloud AI bill before you buy a GPU.

Assume you will run both

Because everything speaks the same OpenAI-compatible dialect, routing by job is a base-URL decision rather than a rewrite. Send the high-volume, low-stakes half locally and the rest to a hosted model, from one codebase. Which hosted model belongs on the far side is covered in the reverse guide to the summer 2026 lineup, and the sizing question is in estimating what fits in your VRAM.

Connecting it to 3D design work

A concrete case from the bench. Parametric CAD is text, which means a tool that writes and rewrites a parameter set is exactly the fixed-shape work local models handle well. Wrap the geometry generator as an MCP tool once and it is callable from whichever application you happen to be in. The practical side of driving OpenSCAD this way is in OpenSCAD and LLMs in practice.

The other half is the file cabinet. Print logs, material notes, failed-print photographs — exposing those as Resources rather than pasting them into a prompt keeps the context small and the answers grounded in what you actually recorded.

Summary — open both passages

• A chat window is an entrance. Real work needs the model callable from outside, and able to call outside.
• Every major runtime speaks an OpenAI-compatible endpoint; Ollama publishes exactly where its support stops.
• In tool calling, your program executes — the model only states intent, which is where validation belongs.
• Start with single calls, widen to parallel, use multi-turn only where dependencies force it.
• No official list of tool-calling-capable models exists; check the model card yourself.
• MCP is JSON-RPC 2.0 over Stdio or Streamable HTTP, with Tools, Resources and Prompts as server primitives.
• In protocol version 2026-07-28, Sampling and Logging are deprecated and Elicitation is current. Check the version before you design.

Sources

• Ollama — OpenAI compatibility (supported endpoints and documented gaps)
• llama.cpp — llama-server README (OpenAI-compatible routes)
• LM Studio — OpenAI Compatibility Endpoints (default port 1234)
• Model Context Protocol — Specification 2026-07-28

ブラウザだけでできる本格的なAI画像生成【ConoHa AI Canvas】

swiftwand

X

AIを使って、毎日の生活をもっと快適にするアイデアや将来像を発信しています。 初心者にもわかりやすく、すぐに取り入れられる実践的な情報をお届けします。 Sharing ideas and visions for a better daily life with AI. Practical tips that anyone can start using right away.