llm-swarm is a local-first mixture-of-models host: a shared KV pool keeps one master record per conversation while an intake model streams the answer and specialist models fan out in parallel — routed by live queue depth and simple if/then rules. Nvidia Dynamo's KV pool meets Switchyard's router as a desktop app.
What it is
Point it at any OpenAI-compatible endpoint — local GGUF/EXL3 servers, LM Studio, vLLM, or paid APIs. The host composes them into one routing brain, and your harnesses (Codex, Cursor, Hermes, Claude Code…) see a single endpoint.
One master record per conversation on the intake model — specialists receive
just enough context, so big slow models load once and stay warm.
Intake streams the answer; sub-tasks fan out concurrently to specialists (vision, coding, writing, security). Judge runs on a concurrent stream.
If/then rules on difficulty, intent, and queue_depth keep your
expensive hardware saturated without starving latency-sensitive intake.
How a request flows
$ swarm_run "Refactor auth middleware, then caption this screenshot" intake ornith 35B → streams plan + prose (concurrent judge stream) fanout glm-5.3 353B → code slice, just enough context, warm on 2× GPU fanout ornith vision → screenshot caption, pinned by intent=vision overflow qwen3.8 27B → cheap slices shed here when queue_depth ≥ 4 merge kv-pool → one master record, TTFT unchanged
worker_queue_depth ≥ 4 → shed, difficulty pins, idle-boost keep-busy rules — you decide which hardware stays fed.Get it
Install, accept your invite, and connect — your first green swarm_run in under two minutes.
curl -fsSL https://latest.llm-swarm.com/install.sh | bash