Ollaya runs open decision models locally, with an MCP server

Ollaya, an open-source runtime from the ollaya-dev organisation, launched in beta with a Show HN-style thread on 25 September 2026. It serves "decision models" locally, the way Ollama serves LLMs, and ships an MCP server and an agent skill so coding agents can call those models directly.

Ollaya is an Apache-2.0 runtime that serves decision models: small models that answer typed choice, score and yes/no questions in a single forward pass, on your own machine, without generating text. Its ollaya mcp command exposes them to Claude Code, Claude Desktop and Cursor as tools, so an agent can classify a request or gate an action in milliseconds.

What changed

Ollaya reads a "state" (a message, an email, any JSON) plus typed questions and returns calibrated probabilities. The Ollaya homepage shows the agent use case first: a request to fix a README typo paired with the command git push --force origin main.

The model answered action: block (0.53), on_task: no (0.75), risk 1.23 out of 2 (0.15) and destructive: yes (0.90). Ollaya says decider:2b produced that in 178 ms on an RTX 4090.

• Question types. choice (2 to 255 options), score (2 to 10 ordered levels) and noul (the probability that a statement holds), per the Laya model page.
• Models. Laya from Convai Innovations, decider from Mapika (built on Qwen3.5), plus NLI, GLiClass, Kev, Von and Qwen3Guard. Ollaya publishes only small ONNX graphs. Weights come from each author's Hugging Face repository, pinned to a commit and checked against sha256.
• MCP server. ollaya mcp speaks stdio and exposes four tools: decide, list_models, show_model and pull_model. In Claude Code the install is claude mcp add ollaya -- ollaya mcp. ollaya mcp --http serves streamable HTTP at http://127.0.0.1:11436/mcp and accepts only loopback hosts by default, per the agents docs.
• Presets. triage, email, guard, moderation, router and agent, available as MCP resources at ollaya://presets/<name>.
• Skill. ollaya-decisions is a SKILL.md covering when a typed decision beats reasoning in text, how to write questions and what confidence thresholds to act on. Install it with npx skills add ollaya-dev/ollaya --skill ollaya-decisions.
• Compatibility. /v1/systemone and /v1/models mirror the wire format of TypeSafe's hosted Jev API, and the official TypeSafe Python SDK 0.7.1 works after three environment variables are set. Ollaya states it is not affiliated with TypeSafe.

Why it matters for agent builders

A local decision call removes a language-model round trip from routing and safety checks. The docs pitch one before every routing, triage, moderation or escalation step. Ollaya reports 8.1 ms for laya:multilingual and 9.6 ms for laya:en on a five-question request on an RTX 4090. The Laya model page lists 16.3 ms and 9.1 ms for the same models in a separate benchmark, so treat the range as the honest answer. The decoder-based decider models are slower, at 155 ms (0.8b) and 190 ms (2b) at the median.

The cost model also changes: no per-token fee, and the server listens on 127.0.0.1 by default. If you already list guardrail or routing tools on aiagentslisting.com, a local classifier is a different category from an LLM-as-judge.

Accuracy is the constraint. Ollaya's own numbers put decider:2b at 0.591 on its typed-decisions set (decider:0.8b at 0.506), and the page warns the labels have low annotator agreement. laya:typed-decisions scores 0.766 against 0.727 published for Jev 1.13, but on Banking77 choice questions with many options, Laya scores 0.425 against 0.870 for Jev. The docs' advice is to measure on your own data before moving production traffic.

What to watch

The example shows the risk: a force push was flagged destructive: yes at 0.90, yet the top-level block answer carried only 0.53. Raw checkpoints are over-confident unless refit, and laya:multilingual ships without refit temperatures, so any threshold that gates a shell command needs calibrating on your own labelled data first. Ollaya supports that through a CALIBRATION setting in a Modelfile.

Hacker News commenters also questioned the framing. ranyume and hbrn argued that "decision model" is a new name for text classification. The author, posting as cobanov, said smaller open models underperform Jev on complex queries and that larger ones are planned, including GGUF LLM-based decision models via llama.cpp. On the hardware side, Apple silicon runs CPU only, and GPU acceleration needs an NVIDIA card with driver R580 or newer.

Key takeaways

• Ollaya serves open decision models locally under Apache-2.0 and was in beta at launch on 25 September 2026.
• ollaya mcp gives Claude Code, Claude Desktop and Cursor four tools, and the ollaya-decisions skill teaches agents when to use them.
• Ollaya reports 8.1 to 9.6 ms for Laya on an RTX 4090 and 155 to 190 ms for the more accurate decider models.
• The best decider model scores 0.591 on Ollaya's typed-decisions set, so calibrate and test before letting it gate real actions.
• The /v1/systemone endpoint follows TypeSafe's Jev API, so existing Jev client code can point at a local server.

FAQ

What is a decision model?

A decision model is a small model that reads some text or JSON plus typed questions and returns a probability for each answer in one forward pass, without generating text. Ollaya supports choice, score and yes/no question types, and each answer carries calibrated probabilities you can threshold.

How do I add Ollaya to Claude Code?

Install Ollaya, then run claude mcp add ollaya -- ollaya mcp. The server exposes decide, list_models, show_model and pull_model. To teach the agent when to call them, add the skill with npx skills add ollaya-dev/ollaya --skill ollaya-decisions. Claude Code and Cursor use the same ollaya mcp command.

Does Ollaya need a GPU?

No. Every model runs on the CPU, but the decider models are slow there and decider:2b needs about 8 GB of memory at fp32. An NVIDIA GPU on Linux, in WSL 2 or in Docker (driver R580 or newer) brings requests down to milliseconds. On Apple silicon, models run on the CPU only.

Source: Ollaya Discussion: Hacker News

Compare local guardrails and routing tools with hosted ones in the MCP servers directory.