🔥 Hot Repo: Run the World's Largest AI Model on Your 4GB GPU

One-liner — AirLLM runs trillion-parameter models like Kimi K3 2.8T on a single 4GB GPU by loading only one transformer layer at a time, trading throughput for accessibility. - Repo: lyogavin/airllmhttps://github.com/lyogavin/airllm - Stars: ⭐ 38,012 +1,004 today - Language: Python - License: Apache-2.0 --- What It Does AirLLM is a Python library that makes large-model inference possible on consumer hardware by keeping only a single transformer layer in GPU memory at a time. Instead of loading the full model, it streams weights from disk layer by layer during a forward pass. For MoE models like Kimi K3, it goes further: it loads only the subset of experts each token actually routes to, so even a 2.8T-parameter model fits in 3.72GB of VRAM — measured on a single RTX 6000 Ada. Why It's Blowing Up July 2026 brought the repo's biggest milestone: support for Kimi K3, currently the largest open-source model in existence at 2.8 trillion parameters. Running it normally requires distributed clusters with terabytes of GPU memory. AirLLM does it on one consumer card by exploiting the MoE architecture — Kimi K3 holds 896 experts per layer but only routes each token to 16 of them, so the active footprint per forward pass is a fraction of the full weight file. The September 2026 update pushed the concept further by adding training support. You can now fine-tune Qwen3.8-Flash-Next 125B under 6GB — a feat previously impossible outside enterprise GPU farms. Frozen weights stream in layer by layer while small LoRA adapters stay resident on the GPU, keeping peak VRAM independent of model size. The tradeoff is speed: Kimi K3 generates roughly one token every 292 seconds on an RTX 6000 Ada. The Hacker News thread was split — some dismissed it as impractical, others called it the only path to running frontier-size models without cloud bills. For researchers and fine-tuners who need a 100B+ model to evaluate or train rather than serve in production, that tradeoff often makes se