WEKA Launches Tech to Boost GPU Utilization
New storage technology offloads GPU KV cache to NVMe flash memory to significantly increase AI processing capacity.
Updated on Oct. 16, 2026 in Data Centers
Live Poll
Is now a good time to bring more AI infrastructure management in-house for your organization?
WEKA released a storage technology that moves GPU KV cache to central NVMe flash memory, addressing the issue where systems must recompute history due to memory limits. This optimization allows data centers to support more users per GPU by freeing up compute cycles.
Why it matters
Organizations face significant constraints on budget, power, and data center space while GPUs currently consume over 80% of AI infrastructure budgets. By increasing GPU utilization, this technology helps companies maximize their existing hardware investments.
WEKA's Augmented Memory Grid improves GPU utilization from 30% to as much as 95% by offloading cache. This shift helps reduce the 60% of compute cycles often wasted on recomputing processed context.
The players
WEKA
The company provides high-performance data storage solutions designed to accelerate AI and machine learning workloads.
Google Cloud
This major cloud service provider manages extensive global data center infrastructure that hosts AI compute resources.
The details
The Augmented Memory Grid captures KV cache from GPU servers and stores it centrally, allowing sessions to move between different GPU hosts. By using NVMe flash storage instead of server DRAM, the system enables session resumption while lowering costs.
Timeline
December 2026 is the deadline for Google Cloud GPU reservations to ensure availability.
The Tech Race
This development represents a shift away from strictly GPU-bound memory limitations toward a more flexible, central storage architecture. It positions storage providers to compete more directly in the AI infrastructure stack, challenging the current GPU-centric AI infrastructure model.
Engineers and developers can expect increased flexibility to move sessions between GPU servers, potentially reducing wait times for AI compute tasks. This could lead to more efficient workflows when managing large-scale AI models within a constrained hardware budget.
The takeaway
By moving cache to central storage, developers can drastically reduce wasted compute cycles spent recomputing data. Adopting such storage-offload techniques is becoming essential as firms look to extend the capacity of their existing hardware fleet.
What happens next
Organizations must secure Google Cloud GPU reservations by December 2026 to avoid potential shortfalls in on-demand capacity.
Further reading
Learn more about the latest innovations in Data Centers infrastructure.
Source note: This article includes information reported by VentureBeat.
Live Poll
Is now a good time to bring more AI infrastructure management in-house for your organization?
