HyMo

A 434M-active / 1.13B-stored hybrid LLM: Gated Delta Net (linear attention) × MLA (full attention) + Asymmetric MoE. Pre-trained from scratch on 30B tokens, FSDP-2, custom Triton GDN kernel. Flagship of the CoreProjects portfolio.

GitHub

Official source

Start locally with the CLI guide