Running DeepSeek-V4-Pro Locally: 1.6T MoE Expert Offloading on Apple Silicon

DeepSeek-V4-Pro is MIT licensed, so it is legally yours to run. That is not the same as it being physically possible to run on your Mac. The model is a 1.6 trillion parameter Mixture of Experts with 49 billion active parameters per token, and those two numbers pull in opposite directions when you try to serve it locally. This post does the arithmetic so you can stop guessing.

The two numbers that decide everything

A Mixture of Experts model has a total parameter count and an active parameter count, and they govern different resources. Active parameters, 49B here, determine how much compute runs per token, which is why a sparse model with 49B active feels roughly as fast to decode as a dense 49B model rather than a dense 1.6T one. Total parameters, 1.6T here, determine how much has to be in memory, because at any given step the router can select any expert, so all of them must be resident or reachable. We covered this split in depth in the total versus active parameter memory analysis, and DeepSeek-V4-Pro is the case that makes it concrete.

Do the memory math. At 4-bit quantization, a rough rule is half a byte per parameter plus overhead, so 1.6T parameters land somewhere near 800GB just for weights, before the KV cache for a long context. There is no single Apple Silicon machine with 800GB of unified memory. The M5 Max tops out well below that, and even an M5 Ultra Studio configured to its ceiling does not reach it. So the full model, at any quantization you would actually want to use, does not fit in unified memory on a Mac. That is the honest answer, and no amount of clever configuration changes the physics.

Model footprint (4-bit approx)	Weights in memory	Fits on M5 Max?	Fits on M5 Ultra Studio?
DeepSeek-V4-Pro, full 1.6T	~800GB	No	No
235B class MoE	~120GB	Marginal	Yes
100B to 120B class MoE	~60GB	Yes	Yes
30B to 49B dense or small MoE	~20GB to 28GB	Yes	Yes

Expert offloading trades one wall for another

The standard escape hatch for a model too big for memory is offloading: keep the hot path in fast memory and push the rest to slower storage. For a dense model you offload layers. For an MoE you can offload experts, keeping the router and shared parameters resident and streaming individual expert weights from SSD as the router selects them. Both llama.cpp and MLX based stacks support variants of this.

It works, and it is slow, because you have swapped a memory wall for a bandwidth wall. Unified memory on an M5 Max delivers hundreds of gigabytes per second. A fast SSD delivers a few gigabytes per second, an order of magnitude or two less. Every token that routes to an expert not currently resident pays an SSD read on the critical path, and because routing changes token to token, the penalty is not a one time load, it recurs. The result is decode throughput that can drop from tens of tokens per second to low single digits, with a fat latency tail whenever a cold expert is selected. If you have read our work on inter token latency and tail jitter, this is the same problem in a crueler form: the tail is now set by disk, not by kernels.

A settings sketch for llama.cpp style expert offload looks like this, expressed as the server options you would set:

llama-server expert offload settings:
  model:           deepseek-v4-pro-Q4_K_M.gguf
  n_gpu_layers:    999
  override_tensor: "\.ffn_.*_exps\.=CPU"   # route expert FFN tensors off the GPU
  ctx_size:        8192
  threads:         12

That pattern keeps attention and shared weights on the GPU and routes the expert feed forward tensors to CPU or disk backed memory. It runs. Whether it runs fast enough to be useful depends entirely on how often your prompts hit cold experts, which you have to measure on your own traffic.

What actually runs well locally

The productive version of the question is not "can I run the 1.6T model" but "what is the largest capable model that fits in my memory budget and decodes at a usable speed." On an M5 Max that answer is typically a 100B to 120B class MoE at 4-bit, or a 30B to 49B dense model with room to spare for context. On an M5 Ultra Studio you can comfortably serve a 235B class MoE. Those fit fully in unified memory, so they get the full memory bandwidth and there is no SSD in the decode path.

Apple Silicon has also gotten materially faster at this. Apple's January 2026 research on the M5 Neural Accelerators showed, for a Qwen3-14B 4-bit model in MLX, time to first token about 4.06 times faster than the M4 and token generation about 1.19 times faster. That helps the models that fit. It does nothing for a model that cannot fit, because you cannot accelerate a weight that is sitting on an SSD. Runtime choice matters here too, and our GGUF versus MLX quantization comparison covers which format to reach for on which size class.

When local is the wrong tool

Here is the decision rule. If you need DeepSeek-V4-Pro specifically, at its full capability, on long contexts, you want the hosted API, not your Mac. The five times cheaper output pricing relative to comparable models makes the API economically sensible for exactly the token heavy work the full model is best at, and you skip the entire offloading problem. Local inference earns its keep when the driver is privacy, offline operation, per token cost at high volume on a smaller model, or latency for short prompts, and when a 30B to 235B class model meets your quality bar. If the requirement is the frontier 1.6T model itself, unified memory is not the place for it yet.

When this applies to your stack

Teams reach for a giant open weights model because the license says they can, then discover the hardware says otherwise. The useful move is to separate the two questions. Decide what quality you actually need, find the smallest model that clears it, and size hardware to run that model fully in memory rather than heroically offloading a model that will never be fast. For most on device and edge workloads that lands you on a mid sized MoE that fits cleanly, with the frontier model called through an API for the rare tasks that genuinely need it.

Contra Collective builds this kind of hybrid AI infrastructure: local models where privacy and cost justify them, hosted frontier models where capability demands them, and a routing layer that sends each request to the cheapest tier that can serve it. If you are trying to figure out what belongs on your own silicon and what belongs in the cloud, that is the work we do.

FAQ

Can I run DeepSeek-V4-Pro on an M5 Max? Not the full model. At 4-bit it needs roughly 800GB of memory for weights alone, far beyond any single Apple Silicon machine. You can attempt expert offloading to SSD, but decode speed collapses to low single digit tokens per second with a heavy latency tail.

Why does a 49B active model still need so much memory? Because active parameters set compute per token, not memory. The router can select any of the 1.6T total parameters at any step, so all experts must be resident or reachable. Total parameters set the memory footprint, active parameters set the speed.

What is the largest model I should actually run on Apple Silicon? On an M5 Max, a 100B to 120B class MoE at 4-bit or a 30B to 49B dense model, both of which fit fully in unified memory. On an M5 Ultra Studio, a 235B class MoE. Keeping the whole model in memory avoids the SSD bandwidth wall entirely.

Does expert offloading make large MoE models practical locally? It makes them possible, not practical, for most workloads. Every token that routes to a non resident expert pays an SSD read on the critical path, and routing changes token to token, so the penalty recurs and the latency tail is wide.

When should I just use the API instead? When you need the full frontier model on long contexts. DeepSeek-V4-Pro is roughly five times cheaper per output token than comparable models, so hosting it via API is economical for the token heavy work it is best at, and you avoid the offloading problem completely.