NVIDIA Redesigns AI Memory Flow to Ease Bottlenecks
NVIDIA outlined changes across GPUs, CPUs, storage and software to reduce memory pressure in large AI systems.
Updated on Nov. 17, 2026 in Artificial Intelligence

NVIDIA outlined a plan to ease memory pressure in AI systems by redesigning how data moves across GPUs, CPUs, storage and software. The company tied these architecture changes to throughput-per-dollar performance claims across its GPU platform generations.
Why it matters
Frontier models with trillions of parameters and agentic systems preserving up to 3 million tokens of context increase demand for high-bandwidth memory, which is among the scarcest and most expensive components in advanced AI hardware. Reducing that pressure through architecture and software changes could affect how efficiently AI infrastructure is built and operated.
NVIDIA's NVL72 systems can distribute model weights across up to 72 GPUs, and its NVFP4 4-bit format reduces space needed for weights and KV cache data. The company's throughput-per-dollar figures come from a single workload, and independent results across other workloads were not provided.
The players
NVIDIA
NVIDIA is a semiconductor company whose GPUs are widely used to train and run large AI models.
The details
NVIDIA's Dynamo software moves inactive context out of GPU memory into CMX context memory storage, using KV Cache Reuse to manage cache across GPU memory, CPU memory and storage as one system-wide resource. Storage partners listed for CMX include Cloudian, DDN, Dell Technologies, Everpure, HPE, IBM, NetApp, VAST and WEKA, while the Vera CPU provides large LPDDR memory linked to Rubin GPUs through NVLink-C2C.
Timeline
• November 17, 2026: NVIDIA outlined its memory-hierarchy redesign in the announcement.
The Tech Race
The redesign extends NVIDIA's NVL72 rack-scale systems, which distribute model weights across up to 72 GPUs, by moving context management into CPU memory and external storage through Dynamo. It positions NVIDIA's memory architecture against rival AI hardware as demand for high-bandwidth memory rises with longer-context agentic workloads.
For developers and organizations running large AI models, memory efficiency affects how much hardware and cost is needed to serve long-context and agent-style applications. The practical effects on pricing and availability for end users were not addressed in the announcement.
The takeaway
Memory capacity and bandwidth are becoming the main constraint on AI system design, so efficiency gains in data movement can matter as much as raw compute. Readers following AI infrastructure costs should watch whether independent benchmarks confirm the throughput claims.
Further reading
For broader context on the hardware behind these systems, see the Artificial Intelligence section.



