$AMD · AMD MI355X HBM advantage may close gap with prefix caching but trails Nvidia today

episode briefing

extracted intelligence

SemiAnalysis

Bryan Shan x Cameron Quilici | Researcher Conversations at GTC

now viewing▶ 6:43

AMD MI355X HBM advantage may close gap with prefix caching but trails Nvidia today

2026-04-07 · 3 company · 3 thematic

sentiment

1 bull0 bear2 neu

speakers

bryan shan

Co-host of SemiAnalysis podcast and co-developer of InferenceX, an open-source AI inference benchmarking suite. Attended GTC 2025 and works directly with Nvidia and AMD engineers on benchmark optimization.

cameron quilici

Co-host of SemiAnalysis podcast and co-developer of InferenceX. Focuses on agentic benchmark design, KV cache offloading strategies, and real-world inference performance modeling.

quote

“AMD MI355X versus B200, MI355 has one and a half times the HBM band capacity. So with prefix caching, right, like we'd expect it to to do to do better possibly.”

—cameron quilici

now playing · $AMD

$NVDA···bullish· highbryan shan

SemiAnalysis benchmarks confirm Nvidia inference leadership across cost and throughput

Independent open-source InferenceX benchmarks show Nvidia chips achieve highest throughput and lowest cost per token across the board, with Nvidia's customer intimacy enabling opt…

$GROQneutral· lowbryan shan

Groq LPU strengths recognized in disaggregated inference but Nvidia acquisition claim is inaccurate

Groq's LPU architecture surpasses GPUs for specific disaggregated inference tasks like fast interactivity, leading to co-packaging with Nvidia's Rubin system, though the claim tha…

$AMD···neutral· mediumcameron quilici

AMD MI355X HBM advantage may close gap with prefix caching but trails Nvidia today

AMD's MI355X offers 1.5x HBM capacity versus Nvidia's B200, which could enable better prefix caching performance in multi-turn workloads, but current InferenceX benchmarks show Nv…