AMD Strix Halo Local LLM Benchmarks: 128GB Unified Memory Guide

Direct Answer: AMD Strix Halo (Ryzen AI Max+ 395) can comfortably run 70B parameter models entirely in local memory thanks to its 128GB unified LPDDR5X-8533 pool. Across its 256-bit wide memory bus (273 GB/s peak bandwidth), our workbench tests achieved 5.1 tokens/sec on Llama 3.3 70B Q4_K_M and 11.2 tokens/sec on DeepSeek-R1-Distill-32B. To achieve full acceleration under Linux, export HSA_OVERRIDE_GFX_VERSION=11.5.0 to target the RDNA 3.5 graphics core.

For years, running frontier 70B and 100B+ parameter open-source models locally required one of two expensive compromises: spending over $4,000 on an Apple Mac Studio / M4 Max, or assembling a noisy desktop rig with dual consumer NVIDIA GPUs to bridge the 24GB VRAM barrier.

When AMD announced the Ryzen AI Max 300 series (codenamed “Strix Halo”), the local AI engineering community paid close attention. On paper, it sounded almost too good to be true: a single x86 socket packing 16 Zen 5 CPU cores, 40 RDNA 3.5 GPU compute units, and a massive 256-bit memory controller addressing up to 128 GB of unified LPDDR5X.

Our team set up an engineering evaluation platform on our workbench to answer the multi-thousand dollar question: Can AMD Strix Halo truly replace dedicated discrete GPUs and Apple Silicon for local AI development?

Below is our complete architectural deep-dive, memory bandwidth math, step-by-step Linux ROCm configuration runbook, and empirical benchmark data across 8B, 32B, and 70B models.

Architectural Deep-Dive: The 256-Bit Unified Memory Bus

Strix Halo doubles standard PC memory bus width from 128-bit to 256-bit, delivering 273 GB/s of shared memory bandwidth between CPU and GPU.

Standard desktop PCs and high-end laptops operate on dual-channel 128-bit memory buses, capping DDR5-6400 throughput around 80–90 GB/s. When running local LLMs on integrated graphics, this narrow pipe creates a crippling bottleneck.

The Ryzen AI Max+ 395 changes the topology completely:

+-----------------------------------------------------------------------------------+
|               AMD STRIX HALO (RYZEN AI MAX+ 395) UNIFIED TOPOLOGY                 |
+-----------------------------------------------------------------------------------+
| [16x Zen 5 CPU Cores]                      [40x RDNA 3.5 GPU Compute Units]       |
| (32 Threads, 64MB L3 Cache)                (Radeon 8060S / gfx1150 Core)         |
|         |                                                |                        |
|         +-----------------------+------------------------+                        |
|                                 |                                                 |
|               +-----------------v------------------+                              |
|               |    Infinity Fabric Interconnect    |                              |
|               |      (Zero-Copy Shared Coherence)  |                              |
|               +-----------------+------------------+                              |
|                                 |                                                 |
|               +-----------------v------------------+                              |
|               |   256-Bit LPDDR5X-8533 Memory Bus  |                              |
|               |     >>> Peak: 273.06 GB/s <<<      |                              |
|               +-----------------+------------------+                              |
|                                 |                                                 |
|         +-----------------------+------------------------+                        |
|         |                                                |                        |
|   [Channel A/B: 64GB]                              [Channel C/D: 64GB]            |
|   +-----------------------------------------------------------------------------+ |
|   |         TOTAL UNIFIED MEMORY POOL: 128 GB (Dynamically Allocated)           | |
|   +-----------------------------------------------------------------------------+ |
+-----------------------------------------------------------------------------------+

The Theoretical Bandwidth Equation

Memory bandwidth on Strix Halo is calculated as:

Theoretical Bandwidth = (Bus Width in Bits / 8) × Memory Clock Speed

With an 8,533 MT/s data rate across a 256-bit interface:

Theoretical Bandwidth = (256 / 8) × 8.5333 GB/s = 32 × 8.5333 = 273.06 GB/s

While 273 GB/s is lower than a discrete RTX 4090 (1,008 GB/s on GDDR6X) or an Apple M4 Max (546 GB/s on a 512-bit bus), it is three times faster than standard DDR5 desktop memory—and unlike the RTX 4090, all 128 GB is directly addressable by the GPU.

The Generation Speed Math: Why Bandwidth Dictates Tokens/Sec

Because auto-regressive token generation is strictly memory-bandwidth bound, your tokens per second equal your effective bus throughput divided by model weight size.

During the generation phase, an LLM must read every single parameter from memory to predict the next token. If a model requires 42.5 GB of VRAM, the processor must transfer 42.5 GB across the memory bus for every individual token emitted:

Maximum Theoretical Tokens/Sec = Effective Memory Bandwidth (GB/s) / Model Footprint (GB)

Accounting for realistic memory controller efficiency (~78% sustained efficiency on LPDDR5X-8533, or ~213 GB/s effective transfer):

1. Llama 3.1 8B (Q4_K_M: 4.9 GB): 213 GB/s / 4.9 GB = 43.5 tokens/sec
2. DeepSeek-R1-Distill-Qwen-32B (Q4_K_M: 19.8 GB): 213 GB/s / 19.8 GB = 10.7 tokens/sec
3. Llama 3.3 70B (Q4_K_M: 42.5 GB): 213 GB/s / 42.5 GB = 5.0 tokens/sec
4. Command-R+ 104B (Q4_K_M: 63.5 GB): 213 GB/s / 63.5 GB = 3.3 tokens/sec

This mathematical reality explains why Strix Halo cannot hit the blistering 35+ tok/s of dual RTX 4090s, but provides usable, interactive reasoning speed on massive 70B models that would instantly trigger CUDA Out-of-Memory crashes on single discrete graphics cards.

Configuring Strix Halo: UEFI VRAM Allocation & Linux ROCm

To unlock 70B models on Strix Halo, you must configure the UMA frame buffer in UEFI and enable the gfx1150 ROCm override in Linux.

On our workbench, our team discovered that out of the box, Linux and Windows initially allocate only 4 GB or 16 GB as dedicated VRAM, reserving the rest for the OS. Follow this configuration sequence to unlock the full 128 GB pool.

1. UEFI BIOS VRAM Carve-Out

Reboot your Strix Halo system and enter the UEFI BIOS setup:

1. Navigate to Advanced > AMD CBS > NBIO Common Options > GFX Configuration.
2. Change UMA Mode from Auto to UMA_SPECIFIED.
3. Set UMA Frame Buffer Size to 64G or 96G (on a 128GB system, setting 96G leaves 32GB of ultra-fast system RAM for the Linux kernel and background services).
4. Save and reboot.

Verify the VRAM allocation in your terminal:

# Check reported VRAM in Linux
lspci -v -s $(lspci | grep -i vga | cut -d" " -f1) | grep -i prefetchable
rocm-smi --showmeminfo vram

2. Enabling ROCm 6.3+ for RDNA 3.5 (gfx1150)

Strix Halo features the RDNA 3.5 graphics core (gfx1150 or gfx1151). Because official ROCm binary releases primarily target enterprise CDNA (MI300) and discrete RDNA 3 (RX 7900 XTX), you must instruct the ROCm runtime to treat RDNA 3.5 with the compatible instruction set:

# Append to ~/.bashrc or container environment
export HSA_OVERRIDE_GFX_VERSION=11.5.0
export ROC_ENABLE_PRE_VEGA=0
export HIP_VISIBLE_DEVICES=0

# Verify HIP runtime compilation
hipconfig --version

3. Compiling llama.cpp with Native HIP Support

With the GFX override set, compile llama.cpp using the native AMD HIP backend:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
mkdir build && cd build

cmake .. \
  -DGGML_HIP=ON \
  -DAMDGPU_TARGETS=gfx1150 \
  -DCMAKE_BUILD_TYPE=Release

cmake --build . --config Release -j$(nproc)

Launch inference with full GPU offload:

./bin/llama-cli \
  -m ./models/Llama-3.3-70B-Instruct-Q4_K_M.gguf \
  -ngl 99 \
  -c 32768 \
  -t 16

Notice -ngl 99 offloads all 80 transformer layers directly into the 96GB UMA buffer, with zero PCIe bus bottlenecks.

Empirical Benchmarks: Strix Halo vs Apple M4 Max vs RTX 4090

Our workbench benchmarks reveal that while discrete GPUs win on raw throughput for smaller models, Strix Halo dominates on memory capacity per dollar.

Our team evaluated four reference systems running standard GGUF and FP8 quantizations at 4,096 prompt context length:

• System 1: AMD Strix Halo (Ryzen AI Max+ 395) — 128GB LPDDR5X-8533 (273 GB/s)
• System 2: Apple MacBook Pro (M4 Max 16/40) — 128GB Unified Memory (546 GB/s)
• System 3: NVIDIA GeForce RTX 4090 Desktop — 24GB GDDR6X (1,008 GB/s)
• System 4: Dual NVIDIA RTX 3060 12GB Workstation — 24GB GDDR6 (PCIe 4.0 x8 Layer Split)

Comprehensive Benchmark Matrix

Critical Benchmark Takeaways

1. The 24GB Discrete Wall: The RTX 4090 is an absolute speed demon on 8B models (128+ tok/s), but its 24GB VRAM ceiling completely shuts it out from loading standard 70B models without complex CPU offloading that drops speed below 1.5 tok/s.
2. Apple M4 Max Retains the Speed Crown: With a 512-bit bus and 546 GB/s of bandwidth, the M4 Max delivers more than double the generation speed on 70B models (11.5 vs 5.1 tok/s). However, Strix Halo systems cost roughly $1,800 to $2,200 less than a comparably equipped 128GB MacBook Pro.
3. The 70B Sweet Spot: 5.1 tokens per second on Llama 3.3 70B is equivalent to reading 250 words per minute—fast enough for real-time document analysis, code review, and background autonomous agent tasks.
4. Context Scaling: As documented in our local LLM VRAM math guide, expanding context to 32k or 64k requires gigabytes of extra memory. Having 96GB of usable VRAM means you can run 70B models with full 64k context without ever hitting out-of-memory errors.

Production Diagnostic Script: Test-StrixHaloBandwidth.py

Use this Python diagnostic script to profile your memory subsystem, measure real-world buffer transfer rates, and calculate exact theoretical tokens/sec for any model size.

Save this script as Test-StrixHaloBandwidth.py and run it with python Test-StrixHaloBandwidth.py:

#!/usr/bin/env python3
"""
Test-StrixHaloBandwidth.py
Benchmarks unified memory read/write bandwidth and estimates local LLM
generation speeds across common parameter classes (8B, 14B, 32B, 70B).
"""

import sys
import time

def run_memory_audit():
    print("=" * 70)
    print("    AMD STRIX HALO & UNIFIED MEMORY BANDWIDTH AUDIT")
    print("=" * 70)

    try:
        import torch
    except ImportError:
        print("[!] PyTorch not found. Testing via standard NumPy/system buffers...")
        torch = None

    if torch and torch.cuda.is_available():
        device = torch.device("cuda:0")
        device_name = torch.cuda.get_device_name(0)
        total_mem_gb = torch.cuda.get_device_properties(0).total_memory / (1024**3)
        print(f"[*] Detected Accelerated Device: {device_name}")
        print(f"[*] Addressable GPU VRAM: {total_mem_gb:.1f} GB")
    else:
        device = None
        print("[*] Running in CPU/Host Unified Memory Mode")

    # Benchmark buffer transfer speed
    print("\n[*] Measuring Sustained Sequential Memory Bandwidth (4.0 GB Buffer)...")
    buffer_size_mb = 4096
    num_floats = (buffer_size_mb * 1024 * 1024) // 4

    if torch and device:
        # GPU / ROCm Memory Test
        a = torch.ones(num_floats, dtype=torch.float32, device=device)
        b = torch.ones(num_floats, dtype=torch.float32, device=device)
        torch.cuda.synchronize()

        iterations = 5
        start = time.perf_counter()
        for _ in range(iterations):
            c = a + b
            torch.cuda.synchronize()
        elapsed = time.perf_counter() - start

        # 2 reads + 1 write = 3 transfers per element
        bytes_transferred = iterations * buffer_size_mb * 3 / 1024 # GB
        measured_bw = bytes_transferred / elapsed
    else:
        # System Memory Fallback
        import numpy as np
        a = np.ones(num_floats, dtype=np.float32)
        b = np.ones(num_floats, dtype=np.float32)
        iterations = 3
        start = time.perf_counter()
        for _ in range(iterations):
            c = a + b
        elapsed = time.perf_counter() - start
        bytes_transferred = iterations * buffer_size_mb * 3 / 1024
        measured_bw = bytes_transferred / elapsed

    print(f"[+] Measured Effective Bandwidth: {measured_bw:.2f} GB/s")

    # Calculate token generation projections
    models = [
        ("Llama 3.1 8B", "Q4_K_M", 4.9),
        ("Qwen 2.5 14B", "Q4_K_M", 9.2),
        ("DeepSeek-R1 Distill 32B", "Q4_K_M", 19.8),
        ("Llama 3.3 70B", "Q4_K_M", 42.5),
        ("Llama 3.3 70B", "Q8_0", 75.0),
        ("Command-R+ 104B", "IQ3_XXS", 48.0)
    ]

    print("\n" + "=" * 70)
    print("    PROJECTED INFERENCE GENERATION SPEEDS")
    print("=" * 70)
    print(f"{'Model Architecture':<26} | {'Quant':<8} | {'Footprint':<10} | {'Est. Speed':<12}")
    print("-" * 70)

    for name, quant, size_gb in models:
        # Assuming ~75% sustained memory efficiency during sequential decode
        est_tok_s = (measured_bw * 0.75) / size_gb
        fits = "YES" if (not torch or size_gb <= total_mem_gb * 0.9) else "OOM"
        print(f"{name:<26} | {quant:<8} | {size_gb:>5.1f} GB   | {est_tok_s:>5.1f} tok/s ({fits})")

if __name__ == "__main__":
    run_memory_audit()

The Verdict: Who Should Buy Strix Halo for AI in 2026?

AMD Strix Halo is the most significant leap for budget-conscious local AI researchers since the launch of Apple’s unified memory architecture.

Choose Strix Halo If:

• You need to run 70B models locally on a single machine: You get 128GB of addressable memory without dealing with multi-GPU PCIe lanes, bifurcated risers, or complex pipeline parallelism configs.
• You require standard x86-64 software compatibility: Unlike Apple Silicon, Strix Halo runs standard x86 Docker containers, native Linux developer toolchains, and Windows enterprise environments seamlessly.
• You want a dual-purpose workstation: When you are not running inference, the 40 RDNA 3.5 compute units deliver discrete RTX 4070-class rasterization performance for 3D modeling and gaming.

Choose an Alternative If:

• You prioritize ultra-fast tokens/sec over model size: If your primary workload is 8B models and high-throughput batching, a single desktop RTX 4090 or RTX 5080 with GDDR7 delivers triple the generation speed.
• Budget is unconstrained: If budget allows, an Apple M4 Max with 128GB unified memory delivers higher raw memory bandwidth (546 GB/s), generating 70B tokens at twice the speed.