deep-fold

Inference stack that keeps LLM weights packed in NF4 for the whole run. Not a new codebook: QLoRA-family NF4, packed once into CHR0. On the GPU each GEMM reconstructs a tile in registers and discards it, a dense layer is never resident. Overflow: two copy slots plus a pinned host tail. Measured on RTX 3080 12 GB.

GitHub

Official source

Start locally with the CLI guide