TickerTain — Make Analysis Fun Again

TickerTain

←

▶ 23:20 · AI Economics & Business Models · Frozen-weight LLM paradigm called unsustainable long-term

episode shorts · 5

Will AGI be one mind? Rich Sutton says no

"Is it Bitter Lesson-pilled?" | Rich Sutton, Oak Lab

Rich Sutton: "I'm not weird. The field is weird"

Rich Sutton: synthetic data is "just a big mistake"

Relearning to walk — what AI is missing | Khurram Javed, Oak Lab

now playing · AI Economics & Business Models

Frontier AI Modelsheadwindscore 8/10rich sutton

Synthetic data generation is a dead end bottlenecked by human expertise

Synthetic data requires human experts to design and validate, creating a fundamental bottleneck. True scaling requires agents that learn autonomously from their own experience in the infini…

AI Agentstailwindscore 8/10khurram javed

Experiential learning agents to replace human-curated synthetic data

The 'big world hypothesis' posits that synthetic data generation is bottlenecked by human expertise; true intelligence requires agents that learn models from their own experience and plan w…

AI Agentstailwindscore 8/10khurram javed

Experiential learning agents must replace human-curated data pipelines

The 'big world hypothesis' posits the world is infinitely complex; synthetic data and simulations are bottlenecked by human expertise. True intelligence requires agents that learn their own…

AI Economics & Business Modelsheadwindscore 7/10rich sutton

Frozen-weight LLM paradigm called unsustainable long-term

Current foundation model labs are locked into a local optimum where models stop learning after deployment; Sutton argues this paradigm cannot achieve general intelligence and will be supers…

AI Agentstailwindscore 8/10khurram javed

Self-improving agents that learn models and plan with them are the missing capability

Current systems lack the ability to learn world models from experience and then plan with those self-discovered abstractions. This model-learning + planning loop is the key to paradigm-shif…

AI Infrastructuretailwindscore 9/10rich sutton

Continual deep learning algorithms needed to replace frozen-weight paradigm

Current LLMs stop learning after pre-training (weights frozen), creating a fundamental gap: they cannot adapt to new experiences or personalize to individual users. Sutton and Javed argue t…

AI Infrastructuretailwindscore 8/10rich sutton

Continual deep learning algorithms target catastrophic forgetting

Sutton and Javed argue current LLMs fundamentally stop learning after training (weights frozen), and propose continual backprop with per-weight step-size optimization and generate-and-test…

AI Infrastructuretailwindscore 8/10rich sutton

Continual learning algorithms could replace static pre-training paradigm

Current LLMs freeze weights after training, preventing true continual learning. Oak Lab's continual backprop algorithm with per-weight step sizes and generate-and-test enables models that c…

AI Hardware & Chip Architecturetailwindscore 7/10khurram javed

Trillion-parameter model at 20W target implies 100x efficiency gains in 5-10 years

Oak Lab targets a trillion-parameter continually learning model running at 20 watts within 5-10 years, relying on two orders of magnitude compute efficiency improvement (Moore's Law) plus a…

AI Hardware & Chip Architecturetailwindscore 7/10khurram javed

Trillion-parameter models at 20 watts targeted within 5-10 years

Oak Lab aims for a trillion-parameter model consuming 20 watts by leveraging two orders of magnitude compute efficiency gains from Moore's Law over 5-10 years combined with algorithmic brea…

Frontier AI Modelsmixedscore 8/10rich sutton

LLMs are a breakthrough in language but only ~25% of intelligence

Sutton acknowledges LLMs as a major scientific breakthrough in neural language use, but argues they represent only a fraction of intelligence (sensory-motor, planning, abstraction). The fie…

←

▶ 23:20 · AI Economics & Business Models · Frozen-weight LLM paradigm called unsustainable long-term

episode shorts · 5

Will AGI be one mind? Rich Sutton says no

"Is it Bitter Lesson-pilled?" | Rich Sutton, Oak Lab

Rich Sutton: "I'm not weird. The field is weird"

Rich Sutton: synthetic data is "just a big mistake"

Relearning to walk — what AI is missing | Khurram Javed, Oak Lab

now playing · AI Economics & Business Models

Frontier AI Modelsheadwindscore 8/10rich sutton

Synthetic data generation is a dead end bottlenecked by human expertise

Synthetic data requires human experts to design and validate, creating a fundamental bottleneck. True scaling requires agents that learn autonomously from their own experience in the infini…

AI Agentstailwindscore 8/10khurram javed

Experiential learning agents to replace human-curated synthetic data

The 'big world hypothesis' posits that synthetic data generation is bottlenecked by human expertise; true intelligence requires agents that learn models from their own experience and plan w…

AI Agentstailwindscore 8/10khurram javed

Experiential learning agents must replace human-curated data pipelines

The 'big world hypothesis' posits the world is infinitely complex; synthetic data and simulations are bottlenecked by human expertise. True intelligence requires agents that learn their own…

AI Economics & Business Modelsheadwindscore 7/10rich sutton

Frozen-weight LLM paradigm called unsustainable long-term

Current foundation model labs are locked into a local optimum where models stop learning after deployment; Sutton argues this paradigm cannot achieve general intelligence and will be supers…

AI Agentstailwindscore 8/10khurram javed

Self-improving agents that learn models and plan with them are the missing capability

Current systems lack the ability to learn world models from experience and then plan with those self-discovered abstractions. This model-learning + planning loop is the key to paradigm-shif…

AI Infrastructuretailwindscore 9/10rich sutton

Continual deep learning algorithms needed to replace frozen-weight paradigm

Current LLMs stop learning after pre-training (weights frozen), creating a fundamental gap: they cannot adapt to new experiences or personalize to individual users. Sutton and Javed argue t…

AI Infrastructuretailwindscore 8/10rich sutton

Continual deep learning algorithms target catastrophic forgetting

Sutton and Javed argue current LLMs fundamentally stop learning after training (weights frozen), and propose continual backprop with per-weight step-size optimization and generate-and-test…

AI Infrastructuretailwindscore 8/10rich sutton

Continual learning algorithms could replace static pre-training paradigm

Current LLMs freeze weights after training, preventing true continual learning. Oak Lab's continual backprop algorithm with per-weight step sizes and generate-and-test enables models that c…

AI Hardware & Chip Architecturetailwindscore 7/10khurram javed

Trillion-parameter model at 20W target implies 100x efficiency gains in 5-10 years

Oak Lab targets a trillion-parameter continually learning model running at 20 watts within 5-10 years, relying on two orders of magnitude compute efficiency improvement (Moore's Law) plus a…

AI Hardware & Chip Architecturetailwindscore 7/10khurram javed

Trillion-parameter models at 20 watts targeted within 5-10 years

Oak Lab aims for a trillion-parameter model consuming 20 watts by leveraging two orders of magnitude compute efficiency gains from Moore's Law over 5-10 years combined with algorithmic brea…

Frontier AI Modelsmixedscore 8/10rich sutton

LLMs are a breakthrough in language but only ~25% of intelligence

Sutton acknowledges LLMs as a major scientific breakthrough in neural language use, but argues they represent only a fraction of intelligence (sensory-motor, planning, abstraction). The fie…