SlipStream Prefetcher Makes Flash Memory Viable for LLMs
A software-hardware co-designed prefetcher from Georgia Tech, NVIDIA Research and Stanford hides flash latency for large language model inference.
Updated on Nov. 17, 2026 in Semiconductors

Researchers at Georgia Tech, NVIDIA Research, and Stanford University published SlipStream, a prefetcher that lets High-Bandwidth Flash, which replaces HBM's DRAM dies with NAND flash, serve large language model inference. The design reached 92.4% of the performance of an ideal High-Bandwidth Flash with HBM-like latency across seven dense models spanning 16 to 800 GB.
Why it matters
Large language model inference demands ever more memory capacity as models and batch sizes grow, and High-Bandwidth Flash offers up to 16x the capacity of HBM at comparable read bandwidth. SlipStream addresses the latency penalty that would otherwise make that capacity impractical.
SlipStream uses a 40 MB staging buffer, against the 320 MB that the H3 tensor-streaming design needs to reach 43.1% of ideal performance. Whether these simulated results hold on physical High-Bandwidth Flash devices has not been established.
The players
Georgia Tech
Georgia Institute of Technology is a public research university in Atlanta, Georgia, known for its engineering and computing programs.
NVIDIA Research
NVIDIA Research is the research arm of NVIDIA, the American chipmaker whose GPUs are widely used to train and run large language models.
Stanford University
Stanford University is a private research university in California, home to a large computer science and electrical engineering department.
The details
SlipStream profiles the repeatable weight-page demand order of data-independent LLM kernels in software, then uses a hardware staging warp as producer and work warps as consumers, retiring each STAGE tracking entry when demand first accesses the page. Conventional cache-line prefetchers would need 20x more in-flight tracking state to cover the flash latency.
Timeline
November 2026: Technical paper published by the SlipStream research team.
The Tech Race
SlipStream builds on the HBM stacks that currently feed AI accelerators, proposing NAND flash as a higher-capacity tier that trades latency for capacity. The approach targets the memory bottleneck that large language models face as model sizes and batch sizes keep growing.
For people who use AI tools, memory designs like SlipStream could eventually mean larger models running on hardware with cheaper, higher-capacity memory. The work remains a research paper, so no products or prices have been announced.
The takeaway
Memory capacity, not only compute, is becoming the limiting factor for large language model inference. Techniques that hide slower memory behind smart prefetching may shape how future AI hardware is built.
Further reading
Explore more semiconductor memory coverage to see how chip designers are tackling AI memory limits.
More information
Read the SlipStream technical paper on arXiv for the full design and evaluation.
Source note: This article includes information reported by Semiconductor Engineering.



