Tiny glue, immense model. Nobody runs the full 2.8T alone — you contribute expert VRAM, the swarm runs Kimi K3 together.
Free routing. llama.cpp + ggml-rpc. Research preview — slow tokens OK.
Like Petals? Same broad idea — many machines, one big model — but MoonTail is deliberately thin glue on llama.cpp + ggml-rpc, targets Kimi K3, and routes volunteers through Cloudflare tunnels instead of a bespoke P2P/DHT stack. How we differ →
One target: moonshotai/Kimi-K3 — 896 experts, 16/token, upstream llama.cpp kimi-k3 arch. MoonTail only pairs sessions and execs llama-cli -rpc -ot.
Reciprocity: volunteer (moontail setup) before you prompt. Backend: operator hosts a free Cloudflare Worker (registry only) — see FAQ.
bash scripts/volunteer-tunnel.sh # once — volunteers
export MOONTAIL_MODEL=/path/to/k3.gguf
moontail setup --accept-terms
moontail prompt "Hello from the Kimi K3 swarm."| Your machine (requester) | Volunteer GPU |
|---|---|
| Backbone, KDA/MLA, router, AttnRes | Routed expert FFN (-ot → RPC) |
llama-cli |
rpc-server @ 127.0.0.1 only |
| Queue + session | cloudflared tunnel out |
flowchart LR
MT[moontail] --> W[Cloudflare Worker]
LL[llama-cli] -->|ggml-rpc| T[cloudflared tunnel]
T --> RPC[rpc-server]
Vol[moontail-volunteer] --> W
Vol --> RPC
Patterns: config/tensor-overrides.kimi-k3. v1: one rpc client per volunteer — pool depth, not 896-way fan-out.
git clone https://github.com/rockybalboan19/MoonTail.git && cd MoonTail
bash install.sh
export PATH="$HOME/.moontail/bin:$PATH"- Weights — community GGUF or convert with
vendor/llama.cpp/conversion/kimi_k3.py. SetMOONTAIL_MODEL. - Volunteer tunnel —
bash scripts/volunteer-tunnel.sh(free Cloudflare account). - Join —
moontail setup --accept-terms - Prompt —
moontail prompt "…"when/statusshows swarm ready.
Details: docs/KIMI_K3_LAUNCH.md
| Command | What |
|---|---|
moontail setup --accept-terms |
Join swarm — volunteer + register |
moontail status |
Pool size, queue |
moontail prompt "…" |
Queue → session → llama-cli |
moontail |
Interactive shell |
bash scripts/check-loc.sh && bash scripts/check-security.sh
python tools/gate3_depgraph.py config/kimi-k3.example.config.jsonGate 4 needs a local K3 GGUF: MODEL=/path/to/k3.gguf bash tools/gate4_expert_rpc.sh
client/moontail.c + cli.c queue, session, llama exec
server/volunteer.c rpc-server + tunnel supervisor
config/tensor-overrides.kimi-k3
vendor/llama.cpp/ inference (submodule)
Control plane: private MoontailAI repo (Cloudflare Worker). URL in config/official-worker.url.
MIT — LICENSE. Kimi K3 weights: Moonshot license.
