mla-from-scratch

Multi-head latent attention from DeepSeek-V2: low-rank KV compression, decoupled RoPE, and absorbed inference verified equal to the naive form. 14.22x cache reduction.

GitHub

Official source

Start locally with the CLI guide