ds4 runs DeepSeek V4 locally and serves Claude Code

ds4 (DwarfStar 4) is a C inference engine for DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next on Metal, CUDA and ROCm. Its ds4-server exposes OpenAI and Anthropic compatible endpoints, so Claude Code and Codex CLI can connect to a model running on localhost. The MIT licensed project calls itself beta quality.

The project, written by Salvatore Sanfilippo (antirez), drew 182 points and 45 comments on Hacker News on 2 October 2026.

What it does

The project is deliberately narrow. The repository describes it as "not a generic GGUF runner": it loads the GGUF files the project itself produces, and it is built around a short list of models. Three programs ship with it: ./ds4 for a chat CLI, ./ds4-server for local APIs, and ./ds4-agent for persistent coding sessions.

The server exposes GET /v1/models, POST /v1/chat/completions, POST /v1/completions, POST /v1/responses (used by Codex) and POST /v1/messages (the Anthropic Messages format). The model name in a request is a compatibility alias. The GGUF loaded at startup is the resident model, and a request name does not swap weights.

Claude Code and Codex

Claude Code speaks the Anthropic wire format, which the server implements at /v1/messages. The server API docs include a wrapper script for Claude Code that unsets ANTHROPIC_API_KEY, sets ANTHROPIC_BASE_URL to http://127.0.0.1:8000, sets ANTHROPIC_MODEL to deepseek-v4-flash and sets CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC to 1. The Codex CLI connects through a model provider entry with wire_api set to responses.

Tool calls use DeepSeek's DSML format under the hood. The docs say the server replays that format byte-for-byte from cache, and describe this as what makes cached tool-call sessions safe.

Caching and hardware

With --kv-disk-dir, the server keeps session prefixes on disk, keyed by the SHA1 of the rendered prompt prefix. An agent that reconnects after a restart resumes without re-prefilling its history. The README names Metal, the primary target, on Macs with 96 GB or more, and says SSD streaming lets smaller machines run large models. The upstream benchmark table, which the project does not date on the page, lists an M5 Max with 128 GB at 2,048 tokens of context at 790.2 prefill and 39.4 generation tokens per second. At 65,536 tokens the same machine reports 398.5 and 27.6.

Why it matters for agent builders

A local endpoint lets a harness run its tool loop against an open-weight model without a per-token bill. The wrapper defaults to 127.0.0.1, so requests go to the same machine.

Because the docs tie cache reuse of tool-call sessions to byte-for-byte replay, a harness should confirm that its own message history reaches the server unchanged before it relies on the cache. The disk cache matters for long coding sessions, since a restarted agent does not pay the full prefill again.

What to watch

The README says the software is "very fast changing" and to "consider it beta quality", with regressions possible before releases. Model support is "intentionally opportunistic": a model may be removed when a better one arrives. A harness that uses deepseek-v4-flash should pin both the GGUF file and the commit it was tested against.

The README also says the project was developed with strong assistance from AI coding agents, and says so openly.

On the Hacker News thread, commenters disagreed about quantization quality. One said the ds4 quants beat the Unsloth quants in a benchmark they linked. Another said the quantized DeepSeek V4 checkpoint was not very good for them. The thread does not settle the question, so test the checkpoint you plan to use on your own tasks.

Key takeaways

• ds4 runs DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next on Metal, CUDA and ROCm, under the MIT license.
• The ds4-server exposes /v1/messages for Anthropic clients and /v1/responses for Codex CLI.
• The docs wrapper points Claude Code at http://127.0.0.1:8000 with the model set to deepseek-v4-flash.
• With --kv-disk-dir, prompt prefixes persist on disk across restarts, keyed by the SHA1 of the rendered prefix.
• The project calls itself beta quality and says its model list may change.

FAQ

Can Claude Code use ds4 instead of the Anthropic API?

Yes, according to the project's docs. The server implements POST /v1/messages in the Anthropic Messages format, and the docs include a wrapper that points Claude Code at it through ANTHROPIC_BASE_URL. The model answering requests is whichever GGUF the server loaded at startup.

Which models does ds4 run?

The README lists DeepSeek V4 Flash (with an experimental vision model), DeepSeek V4.1 Flash, GLM 5.2 and 5.3, GLM 5.3 Flash, DeepSeek V4 PRO and Qwen3.8 Flash Next. Support is opportunistic, and a model may be removed when a better replacement arrives.

What hardware does it need?

The README names Metal on Macs with 96 GB or more as the primary target and supports NVIDIA CUDA, including the DGX Spark, and ROCm on Strix Halo systems. The project site's hardware table lists 64 GB and up depending on the model, which does not match the README, so check the hardware guide for your exact model before buying.

Developers comparing agent tools that run on their own hardware with hosted ones can browse the coding category on AI Agents Listing.

Source: dwarfstar.sh Discussion: Hacker News