Startup Swaps Costly AI GPUs for 128TB LPDDR6 RAM in 2026

The story behind LPDDR6 RAM and AI startups is quite fascinating. In 2023, a group of former engineers from Google and Meta launched Majestic Labs in Tel Aviv. Their mission? To tackle a pressing issue: the combination of high-bandwidth memory and graphics processors creates a significant bottleneck for enterprise workloads.

Details about their plans come primarily from leaks and supply-chain reports, as nothing has been confirmed officially yet. The enterprise computing sector is paying close attention as hardware designers search for alternatives to those pricey $30,000 graphics cards. Majestic Labs created a system that swaps traditional accelerators for general-purpose CPU architecture combined with high-capacity commodity memory, aiming to break through existing processing limitations.

Startup’s Strategic Shift

The race for hardware in the industry has led to severe supply constraints and rising operational costs for hyperscalers. Majestic Labs kicked off its hardware pivot by moving away from standard accelerator designs to create custom silicon called Ignite AI Processing Units. Instead of depending on power-hungry graphics processors, these custom chips marry low-power general CPU clusters with specialized vector and tensor math engines.

This strategy emphasizes sharing memory across nodes instead of isolating high-speed memory within individual chips. Server nodes connect via custom aggregation chiplets linked with one-meter copper cables, forming a single cohesive memory pool. This setup enables multiple computing units to access data without delays caused by network transfers between separate board sockets.

Cost Comparison: AI GPUs vs. Arm Cores

Traditional enterprise setups usually pair NVIDIA graphics hardware with stacked silicon memory dies, pushing hardware costs into five figures for each socket. By using Arm cores along with RISC-V computing extensions, system builders can lower silicon licensing fees and simplify manufacturing. Commodity chips typically achieve better manufacturing success rates compared to large monolithic graphics dies.

Switching from dedicated graphics hardware to general CPU designs also cuts down on overall system power consumption during idle times. Standard data centers using conventional hardware often face power delivery challenges long before they fill their physical rack space. Moving execution to flexible general-purpose clusters lessens thermal dissipation needs throughout enterprise data halls.

Technical Implications of New Architecture

When processing large language models, typical systems waste time moving data from system memory to processor caches instead of focusing on actual matrix calculations. Engineers refer to this hardware bottleneck as the memory wall, which significantly limits execution efficiency, no matter how fast the processor clock speed is.

To overcome this physical barrier, the Prometheus server architecture incorporates up to 12 AIUs per node. These clusters can access between 8TB and 128TB of unified LPDDR6 RAM across a single coherent address space. A fully loaded 40U rack setup accommodates four servers that consume 120kW, necessitating direct cold-plate liquid cooling rather than traditional forced air.

Advantages of LPDDR6 RAM Over HBM

While high-bandwidth memory provides exceptional throughput per individual die, its physical manufacturing constraints limit maximum memory capacity. In contrast, mobile-grade LPDDR6 RAM offers a lower unit cost and significantly higher capacity potential when aggregated through external chiplets, as discussed in reports on next-generation server architectures.

By replacing costly stacked silicon with aggregated mobile memory, enterprise customers can run large models in active system memory. Systems that once needed multi-rack GPU clusters can now handle extensive inference workloads within a single server enclosure, optimizing budget allocations for high-density environments, as seen with Microsoft’s initiatives.

Source: Techradar