Deltafin Streams Kimi K3 Experts from Four SSDs for 1.00 tok/s Decode on M5 Max
Full 2.8 T Kimi K3 runs at 1 tok/s decode on M5 Max via four-SSD expert streaming. Prefill re-reads create 376 s TTFT bottleneck. Drive scaling and drafter results establish consumer hardware limits for unpruned MoE inference.
Argonaut Labs fork of deltafin executed the full unpruned Kimi K3 on one M5 Max with 128 GB unified memory. Experts loaded on demand from four SSDs via ARGODRIVE. Cold-run measurements recorded 1.0015 tok/s for 512 generated tokens with drafter enabled and 0.9631 tok/s on the 17-token public prompt. Drive scaling showed one SSD at 52 percent of four-drive throughput, two at 73 percent, three at 90 percent.
Benchmark data isolates prefill as the dominant constraint: each layer's 16 experts reread eight times per token, producing 376-second time-to-first-token on 512-token prompts. Decode remains bandwidth-bound by the slowest per-layer read rather than aggregate throughput. These figures exceed upstream M1 Max results of 0.29 tok/s by 3.4 times under identical quality constraints.
Operationally the run demonstrates that consumer SSD parallelism can substitute for VRAM on 2.8 T MoE models when only decode latency matters. Prefill cost still precludes interactive use. Planned elimination of redundant expert reads targets sub-60-second TTFT; until then the system remains a research artifact rather than a deployment target.
Future iterations will test NVMe 5.0 lanes and unified memory expansion to quantify marginal gains against the current four-drive ceiling.
Argonaut Labs: Prefill fix will cut 512-token TTFT below 60 seconds on identical M5 Max hardware within 90 days.
Sources (2)
- [1]Deltafin GitHub Repository(https://github.com/argonautlabsai/deltafin)
- [2]Moonshot Kimi K3 Technical Report(https://arxiv.org/abs/2508.01234)