Nvidia Outlines Memory Strategy for Growing AI
Nvidia describes a combined hardware and software approach to improve memory use as AI models, context windows and agent applications expand.
Updated on Nov. 7, 2026 in Artificial Intelligence
Live Poll
Do you expect AI-powered products to get cheaper for everyday consumers?
Nvidia outlined a memory strategy for growing AI systems that combines GPUs, CPUs, networking, storage and software. The company said Vera Rubin NVL72 reaches up to 30 times GB300 NVL72's cost-adjusted inference throughput.
Why it matters
The strategy addresses growing models, lengthening context windows and spreading agent applications, all of which place heavy demands on memory. Improving memory use is a central constraint on how efficiently AI systems can run.
Nvidia reports Vera Rubin NVL72 reaches up to 30 times GB300 NVL72's cost-adjusted inference throughput, while its per-GPU high-bandwidth memory capacity is about 3% higher than GB300 NVL72's. The company's figures are self-reported, and the specific workloads behind them were not detailed.
The players
Nvidia
Nvidia is a American semiconductor company whose GPUs are widely used to train and run artificial intelligence models.
The details
Nvidia's methods compress model weights and KV Cache using NVFP4, a low-precision format. The approach also moves inactive context to lower-cost storage and distributes work across multiple GPUs to free high-bandwidth memory.
The Tech Race
The memory strategy relies on NVFP4, a low-precision number format, to shrink model weights and KV Cache, extending that compression approach across the full hardware stack. Nvidia positions the Vera Rubin and GB300 generations against its own prior H200 system to show cost-adjusted gains over successive hardware releases.
The efficiency gains described could lower the cost of running AI tools that businesses and developers rely on, potentially affecting pricing for AI services. The strategy targets inference workloads, so the practical effect on everyday users will depend on how quickly these systems reach commercial deployment.
The takeaway
Memory efficiency is becoming as important as raw compute for AI systems, and the design choices behind it shape what those systems cost to run. Readers tracking AI infrastructure costs should watch how these techniques perform outside vendor-reported benchmarks.
Further reading
Explore more on Artificial Intelligence developments shaping how large models run.
Live Poll
Do you expect AI-powered products to get cheaper for everyday consumers?
