newsroom
Semiconductors · Custom GPU kernels and numerical determinism are critical for MoE model RL training stability
now playing · Semiconductors
AI Coding Agentstailwindscore 9/10federico
Application companies should train specialized foundation models on proprietary usage data and harnesses
Cursor demonstrates that owning the full stack — application, harness, user data, and model — enables 10x cost/performance gains by specializing every model weight to the exact environment…
AI Infrastructuretailwindscore 9/10dima
Distributed RL training across heterogeneous global clusters reduces need for massive contiguous GPU clusters
Fireworks' architecture disaggregates training (needs high-bandwidth interconnect) from inference (can run on smaller, heterogeneous, cheaper clusters worldwide), using delta weight synchro…
AI Agentstailwindscore 8/10federico
RL on production environments is essential for long-horizon agent capabilities and tool use
Reinforcement learning against the actual product harness — not toy environments — teaches models correct tool calling, navigation, and self-correction (e.g., self-summarization for context…
Semiconductorstailwindscore 7/10dima
Custom GPU kernels and numerical determinism are critical for MoE model RL training stability
MoE architectures amplify floating-point non-determinism during asynchronous RL (expert routing divergence), requiring kernel-level fixes like router replay and deterministic accumulation o…