rohan anil

Current chain-of-thought scaling spends compute one token at a time, making inference costly and limiting frontier model access to a subset of users; architectural changes that increase computational depth without token-by-token generation are needed to expand the addressable market for AI.

Current separate pre-training (perplexity minimization) and RL (chain-of-thought) pipelines are suboptimal; combining them with second-order optimizers like Shampoo and architecture-optimizer co-design can yield 10x+ compute efficiency gains by aligning training objectives with inference-time computation patterns.

Novel architectures require custom high-performance kernels (e.g., 60x speedup for QR factorization), but current models cannot write them; automating kernel generation via AI-assisted search is the critical inner loop enabling rapid architecture experimentation on modern hardware like B200 GPUs.