stuart

GPU networking remains the primary bottleneck (up to 50% of runtime for Llama MLA prefill). The Parallel Kittens framework identifies three critical tradeoffs — transfer mechanism (copy engine vs TMA vs register instructions), scheduling strategy (intra-SM vs inter-SM overlap), and design overhead (intermediate buffers) — enabling 50-100 lines of device code to match hand-optimized kernels of thousands of lines. Adopted by Cursor (training on tens of thousands of Blackwell GPUs) and Together AI (inference optimization).

Networking consumes up to 50% of runtime for LLM prefill; fine-grained overlap of compute and communication at tile/token granularity is critical. In-network compute offloads collectives to fabric, freeing SMs. NVL72's 72-GPU NVLink domain and future hundreds-GPU scale-up demand new programming abstractions (Parallel Kittens) to exploit hardware without complexity explosion.

Collective operations (all-reduce, all-gather) moving into the network fabric frees GPU SMs for compute. Asynchronous bulk device-initiated networking and tile-granularity overlap (KB not MB) require new programming models. NVL72's 72-GPU domain and future scale-up make networking the primary architectural lever for cluster efficiency.