setheerwagen/mcp-ollama-vllm

Two self-hostable Model Context Protocol (MCP)
servers for running and steering your own local language models: manage an
Ollama/vLLM model machine, and query a local LLM (Ollama, vLLM or an
OpenAI-compatible backend) for language work with schema-validated JSON output.
Self-hosted and privacy-first: the models and their machine stay on your own
hardware, and neither server process reaches out to a public address on its
own. Version 1.0.0, single-operator tool, actively used by the author.
Both servers speak the Model Context Protocol (MCP), so an AI tool such as Claude
or ChatGPT can use them. Each server's own traffic goes only to your own computer
or your local network; a start-up guard rejects public addresses. Installing the
two direct dependencies needs the internet once, at setup time. Fetching a model
is a separate matter: in mcp-modelmanager, calling fetchmodel is a
deliberate, tool-triggered connection from your model machine to a public
model registry, at whatever point you invoke it, not something limited to
setup time. The default source="ollama" makes the machine's own Ollama
daemon pull from registry.ollama.ai for plain model names, or from whatever
other registry a prefixed name such as hf.co/user/repo points at;
source="huggingface" instead reaches Hugging Face directly. switchservice
never reaches either registry itself;
it refuses and asks for fetchmodel first if the model is not already on the
machine. Each server stands on its own: pick one, or run both.
A caveat: the AI tool you drive these servers with (Claude, ChatGPT,
etc.) is itself a hosted service in the usual case. The instructions you give it
and the tool inputs and results it handles pass through that provider, exactly as
any other MCP call would. "Local" here means the models, the model machine and
its data; it is not a claim that nothing you type ever reaches a third party.
Drive them with a local model if that matters to you.
## The two servers
### mcp-modelmanager -> mcp-modelmanager/
Manage your own model machine through an AI tool: read state (GPU memory, disk,
running service, containers), pull and remove models, switch the vLLM inference
service to a different model, build custom Ollama variants and run LoRA training.
There is deliberately no tool for free remote control: every operation is a fixed
template and every parameter first passes an allowlist, checked independently on
both sides. This server ships both halves: the client/server code the AI tool
talks to, and the machine-side scripts, wrapper and network-fence templates that
run on and lock down the model machine.
### mcp-ollama-vllm -> mcp-ollama-vllm/
Call a local model directly from an AI session for the language work: writing,
summarizing, classifying, extracting structured data. Its distinctive feature is
schema-validated JSON output, checked by its own dependency-free validator and
retried automatically on a violation, the same way whether the configured backend
is Ollama or vLLM. It is a model call, not an agent: no tools, no file access, no
shell.
The two are complementary. mcp-ollama-vllm asks a model (read-only, no
management rights); mcp-modelmanager manages the machine. Keeping them