Xiaomi AI Cube Targets Local LLMs With 1.22 TB/s Near-Memory Bandwidth

Xiaomi has shown a prototype AI workstation called the Xiaomi AI Cube, a compact 150W system that combines three of the company’s XRING processors in one machine. For local LLM users, the headline number is 1.22 TB/s of bandwidth on the XRING O100 accelerator, alongside support for large local models and a system design built around three different chips.

The specifications sound impressive, but they also need some careful interpretation. The 1.22 TB/s figure does not appear to describe conventional system RAM bandwidth in the same way we would describe an NVIDIA GPU, AMD Strix Halo system, or Apple unified-memory machine. Xiaomi specifically describes it as “near-memory computing bandwidth” on the O100. That distinction matters if you are trying to estimate real LLM inference performance.

Xiaomi AI Cube Uses Three XRING Chips

The AI Cube prototype combines the XRING O3, O100, and D100 in a single system. Xiaomi describes this as tri-chip collaborative computing. The machine is rated for 150W sustained operation and is intended to run large models locally. Xiaomi also demonstrated a 120B model together with a 3B model, with the system supporting switching between a faster and slower inference mode.

That three-chip architecture is unusual compared with the more familiar local LLM approach of putting most of the workload on one large GPU or unified-memory SoC.

The O3 is the general-purpose SoC. Xiaomi lists a 10-core CPU, a 16-core G2 Ultra NX GPU, an NPU rated at 200 TOPS, and support for LPDDR6. It is manufactured on a 3nm process.

The O100 is the component that matters most for the 1.22 TB/s claim. Xiaomi describes it as a high-bandwidth AI accelerator built on a 6nm process with 3D wafer-level stacked packaging. The chip has 28,672 effective data lines, a 14-core high-bandwidth NPU, a 1.4 µm bonding pitch, and 1.22 TB/s of near-memory computing bandwidth.

The D100 comes from Xiaomi’s automotive work. It includes a 20-core CPU, a 16-core NPU, a RISC-V security core, and 3.13 TFLOPS of vector compute. Xiaomi also associates the D100 platform with local deployment of models up to 200B parameters.

What Does 1.22 TB/s Actually Mean?

This is the main question for anyone looking at the AI Cube as local LLM hardware.

At first glance, 1.22 TB/s sounds enormous for a 150W mini PC. NVIDIA’s DGX Spark, for example, is listed in the supplied material at 273 GB/s of unified-memory bandwidth. On paper, Xiaomi’s number is more than four times higher.

But those numbers should not be treated as directly comparable yet.

Xiaomi calls the O100 figure “near-memory computing bandwidth,” rather than simply system memory bandwidth. The supplied technical material describes vertically stacking AI-focused high-speed memory dies with an NPU die, using hybrid bonding to create a very large number of short physical connections between the two.

That architecture can produce very high bandwidth close to the accelerator without necessarily giving the entire system 1.22 TB/s of access to hundreds of gigabytes of ordinary addressable RAM.

This is why the AI Cube should not yet be described as a 160GB or 200GB machine with 1.22 TB/s of conventional unified-memory bandwidth. The information available in the supplied material does not establish that.

There is possibility that the bandwidth could refer to SRAM or another form of tightly coupled near-memory storage rather than the AI Cube’s main DRAM pool. It is important to mention that Xiaomi uses the Chinese term 近存, approximately “near memory” or “near storage,” instead of ordinary memory terminology.

Until Xiaomi publishes a proper memory architecture diagram, capacity breakdown, and sustained inference benchmarks, the safest interpretation is simple: the O100 has an extremely high-bandwidth local memory interface, but we do not yet know how much memory is attached to that interface or how much of the AI Cube’s total memory can sustain anything close to 1.22 TB/s.

Why Memory Capacity Still Matters More Than the Headline Bandwidth

For local LLM users, memory bandwidth is only useful after the model fits.

A dense 70B model at 4-bit quantization typically needs roughly 35GB just for weights before accounting for KV cache and runtime overhead. A 120B model at four bits is already around 60GB of raw weights. A 200B model is around 100GB before additional memory requirements.

That is why Xiaomi’s large-model claims are interesting.

The company says the AI Cube can deploy large models locally, and its presentation references 120B and 3B dual-model operation. The O100 and D100 material also mentions support for local deployment of models up to 200B parameters. However, there is still uncertainty around exactly where those model weights sit.

There are several conflicting interpretations of the memory layout. A large memory pool could sit with the D100 while the O100 uses a much smaller but much faster near-memory pool. Another interpretation is that memory is distributed asymmetrically across all three chips.

That difference is critical.

If the O100 only has a few gigabytes of very fast near-memory capacity, then 1.22 TB/s could be excellent for accelerating certain parts of inference while the bulk of a 120B or 200B model remains in slower memory attached elsewhere.

If the O100 can directly access a much larger pool at close to that bandwidth, then the AI Cube becomes much more interesting as a general-purpose local LLM machine.

At the moment, the supplied information does not confirm which architecture Xiaomi is using.

1.22 TB/s Does Not Automatically Mean Fast Tokens Per Second

Local inference performance is usually constrained by some combination of memory capacity, memory bandwidth, compute throughput, quantization support, kernel quality, and model architecture.

Large dense models during token generation are often heavily memory-bandwidth bound. That is why high-bandwidth GPUs tend to perform well even when their theoretical compute capability is not fully utilized.

But memory bandwidth alone does not generate tokens.

A very fast memory subsystem can still be paired with an accelerator that does not have enough compute to fully use it.

This is particularly important with the O100 because Xiaomi has disclosed the 1.22 TB/s near-memory number but has not provided a directly comparable FP16, FP8, FP4, or INT4 compute figure for the accelerator in the supplied material.

img

Xiaomi does claim that the O100 can reach around 330 tokens per second in an on-device large-model workload, but the supplied information does not provide enough detail about the model, quantization, prompt length, batch size, or exact workload to turn that into a useful comparison with an RTX 5090, Strix Halo, DGX Spark, or Mac Studio.

Xiaomi AI Cube vs DGX Spark

The AI Cube naturally invites comparison with NVIDIA’s DGX Spark because both are compact AI systems targeting workloads that previously required much larger hardware.

The supplied comparison lists DGX Spark with 128GB of unified memory, 273 GB/s of memory bandwidth, approximately 1 PFLOP of sparse FP4 AI compute, and roughly 140W power consumption. The Xiaomi prototype is listed at 150W, while the O100’s near-memory interface reaches 1,220 GB/s.

The important difference is that DGX Spark’s architecture is much easier to understand.

Its 128GB unified-memory capacity and bandwidth are known quantities. Xiaomi’s design appears to use several memory domains spread across three processors, with a very fast near-memory subsystem attached to the O100.

For local LLM builders, that means DGX Spark remains easier to evaluate today. You can roughly calculate what models fit and what memory bandwidth is available to them.

The AI Cube could turn out to be faster in some inference workloads, but the published specifications are not yet sufficient to prove that.

Software Support Could Decide Whether the AI Cube Matters

Hardware specifications are only half of the local LLM story.

For enthusiasts, support in llama.cpp, vLLM, PyTorch, MLX-style frameworks, or another open runtime can matter more than an impressive TOPS number.

Competitive LLM accelerators have struggled when they depended entirely on vendor-specific runtimes and limited model support. If a new model architecture appears and users have to wait months for Xiaomi to update its software stack, the hardware becomes much less attractive to enthusiasts.

At this point, the supplied material does not confirm upstream llama.cpp support, vLLM support, open kernel documentation, or an open accelerator API for the XRING chips.

That is a much bigger unknown than the bandwidth figure.

A 1.22 TB/s accelerator that only works with Xiaomi’s approved model would have limited value for people who constantly test new Qwen, DeepSeek, GLM, and other open-weight releases.

A slower device with a well-supported llama.cpp backend can easily be more useful.

The 120B Plus 3B Model Setup Is Interesting

One part of Xiaomi’s demonstration could make sense for local agent workloads.

The AI Cube runs a 120B model alongside a 3B model and supports switching between fast and slow systems.

That suggests a design where a small model handles lightweight requests while the large model is activated for harder tasks.

For a home server or small office AI system, that could be useful. A small model could handle classification, routing, basic automation, RAG filtering, or simple tool calls while the larger model handles reasoning and generation.

This would also fit Xiaomi’s broader phone, vehicle, and smart-home ecosystem. The supplied material describes the XRING platform as a computing base for Xiaomi’s “Human-Vehicle-Home” strategy.

For a local LLM enthusiast, though, the interesting question is whether Xiaomi exposes enough of the hardware to let users build their own routing and inference stack rather than forcing them into Xiaomi’s software environment.

Price Will Determine Whether the Xiaomi AI Cube Makes Sense

There is currently no confirmed price in the supplied material, and the AI Cube is still a prototype.

That means performance-per-dollar cannot yet be calculated.

This is especially important because the local LLM market already has several very different ways to buy memory capacity.

Used RTX 3090 cards remain attractive because multiple cards can be combined to reach 48GB, 72GB, 96GB, or more VRAM while retaining mature CUDA support. AMD Strix Halo systems offer large unified-memory configurations with much lower power consumption. DGX Spark provides 128GB in a compact integrated system. Apple’s higher-memory Mac Studio configurations offer even more unified memory, although inference behavior differs significantly from CUDA systems.

The Xiaomi AI Cube only becomes a serious enthusiast option if three things line up: usable memory capacity, open software support, and price.

If it arrives with enough directly accessible memory for 100B-class quantized models, strong real-world decode speeds, and a price below comparable NVIDIA systems, it could be very interesting.

If the 1.22 TB/s number applies only to a small near-memory cache while most model weights sit behind a much slower memory subsystem, the headline specification will matter far less.

What We Still Need to Know

The Xiaomi AI Cube is one of the more unusual local AI hardware designs shown recently, but it is still too early to rank it against DGX Spark, Strix Halo, Apple Silicon, or multi-GPU builds.

The critical missing information is not another TOPS figure. It is the complete memory map.

We need to know how much RAM each processor has, which processors can directly access which memory pools, what bandwidth is available between the three chips, how much capacity sits behind the O100’s 1.22 TB/s interface, and whether model weights can remain there during inference.

We also need real measurements for prompt processing and token generation with known models and quantization formats.

Finally, Xiaomi needs to clarify the software stack. For the local LLM community, native support in open tools could be the difference between an interesting prototype and hardware people would actually build around.

For now, the 1.22 TB/s figure is worth paying attention to, but it should be read specifically as O100 near-memory bandwidth rather than automatically interpreted as 1.22 TB/s of system-wide unified memory. That distinction clears up much of the confusion around the AI Cube’s specifications and gives a more realistic picture of what Xiaomi has actually announced.