How to Run GLM-5.3-Flash Locally: Hardware & Setup

Z.ai published the GLM-5.3-Flash weights on Hugging Face on 25 August 2026 under an MIT licence — the model that had been serving anonymously as "Ox Alpha" on OpenRouter. The headline spec is 320B total parameters with 18B active, and that second number is why people immediately asked whether they could host it themselves.

They asked the wrong question. Active parameters govern how much compute each token costs; total parameters govern how much memory you need. This page is about the memory, the actual file sizes, and the commands that work today.

If you want to know what the model is — benchmarks, pricing, API behaviour — read the GLM-5.3-Flash complete guide. For the stealth-launch backstory, see our Ox Alpha writeup.

What do you actually need to run GLM-5.3-Flash?

Every number in the table below is either measured from the Hugging Face repository file listing or published by the people who built the quantisation. The "size on disk" column is what you download. The memory floor is higher because you also need KV cache, activations and framework overhead.

Sizes measured from the Hugging Face file listings for zai-org/GLM-5.3-Flash and unsloth/GLM-5.3-Flash-GGUF. Accuracy and memory-floor figures from Unsloth's quantisation analysis.

Two things to notice. First, the official checkpoint is already FP8 — the repo's config.json declares quant_method: fp8 with e4m3 format and 128×128 weight blocks. You are not downloading BF16 and quantising it; 314B of the 321B parameters ship as single-byte values. The vLLM recipe page states the same figure independently: "about 306 GiB for the default native FP8 checkpoint."

Second, quality falls off a cliff below 3-bit. The 1-bit build fits in 100 GB but keeps only 71% of top-1 accuracy. Unsloth's own recommendation for 128 GB devices is UD-IQ3_XXS at 120 GB, which holds 82%.

Why doesn't 18B active parameters mean 18B worth of memory?

This is the single most misunderstood thing about MoE hosting, and it is worth being blunt about.

GLM-5.3-Flash routes each token through 8 of 288 routed experts plus one shared expert. Only those experts do arithmetic, so the per-token compute is roughly that of an 18B dense model — which is why it is fast and why the API is cheap. But the router can pick any 8 experts for the next token, so all 288 must be resident. Nothing about sparsity reduces the resident weight footprint.

The practical consequence: GLM-5.3-Flash is a memory-bound problem, not a FLOPs-bound one. A pile of cheap memory beats a small amount of very fast memory. That is exactly why Apple Silicon and CPU-plus-RAM boxes are viable here in a way they are not for a dense 300B model.

The one place the architecture genuinely helps is KV cache. The 45 language layers are a hybrid: 34 KDA linear-attention layers and 11 DeepSeek-sparse-attention layers using rope-free MLA with a 512-dimension latent. Only those 11 layers keep a paged KV cache. vLLM measures a 14.92M-token KV pool at TP=4 with FP8 weights and FP8 KV — roughly 114× concurrency at 128K context. Long context is genuinely cheap on this model; the weights are the problem.

Can you run GLM-5.3-Flash on a single GPU?

No. Not on any GPU that exists. The smallest useful build is 120 GB and the largest single accelerator you can buy has 192 GB — and even a 192 GB B200 running the 156 GB IQ4_XS build leaves little room for KV cache and activations.

The published multi-GPU recipes both start at four GPUs:

• vLLM: TP4 on one GB200 tray for the FP8 checkpoint; the NVFP4 variant needs Blackwell.
• SGLang: TP4/EP4 on 4× GB300 for the verified NVIDIA recipes; TP8 on a single eight-GPU node for AMD ROCm.

A 4× RTX 6000 Blackwell workstation (4 × 96 GB = 384 GB) is the cheapest all-GPU configuration that clears the FP8 footprint on paper. Note that vLLM does not yet have a sparse-MLA path for Ada (sm_89), so a stack of RTX 4090s is not a supported target — that is an open issue, not a tuning problem.

Can GLM-5.3-Flash run on a Mac?

This is the most interesting answer, and it is yes.

Unified memory means the whole 512 GB of an M3 Ultra Mac Studio is addressable by the GPU, and MoE inference is memory-bandwidth bound rather than compute bound. The measured data point: inferencerlabs' Q9 MLX build runs multimodal inference at ~22.3 tokens/sec using ~332.8 GiB on an M3 Ultra. That is a near-lossless quantisation producing usable interactive speed on a single desktop machine.

Sizing by machine:

• 512 GB M3 Ultra — 8-bit MLX (334 GB) or Q8_0 GGUF (341 GB). Effectively full quality. BF16 at 642.7 GB does not fit.
• 256 GB — 6-bit MLX (255.8 GB) is borderline; UD-Q5_K_XL at 240 GB or UD-Q4_K_XL at 200 GB are the safe picks.
• 192 GB — 4-bit MLX (177.5 GB) or UD-IQ4_XS (156.8 GB). Raise the GPU wired limit with sudo sysctl iogpu.wired_limit_mb first.
• 128 GB — UD-IQ3_XXS (120.4 GB). This is the floor Unsloth recommends for 128 GB devices.
• Under 128 GB — only the 1-bit and 2-bit builds, and the quality loss is real.

One caveat: glm5_next landed in mlx-vlm's main branch on 26 August 2026 and no tagged release carries it yet. The PipeNetwork port documents two genuine numerical bugs in that first port — a missing swiglu_limit clamp affecting all 45 FFN blocks, and an mHC dtype cast — so use a runtime that has those fixes, not an arbitrary checkout.

Can you run it on CPU or with RAM offload?

Yes, and for this model it is less silly than usual. With only 18B parameters active per token, CPU inference moves far less data per token than a dense model of comparable size. Unsloth's guidance treats the requirement as total memory — "RAM + VRAM, or unified memory" — because llama.cpp will happily keep experts in system RAM and stream them.

Build llama.cpp with -DGGML_CUDA=OFF for pure CPU, or keep CUDA on and let the runtime offload MoE layers to host memory. A dual-socket server with 256 GB of DDR5 will run UD-Q4_K_XL; a 128 GB workstation will run UD-IQ3_XXS. Expect single-digit tokens per second on CPU-only, which is fine for batch work and painful for chat. Public tokens-per-second measurements for CPU builds are thin right now, so treat any specific figure you see with suspicion, including ours — we have not measured one we would stand behind.

How do you serve GLM-5.3-Flash with vLLM?

Important caveat first: the vLLM model PR is still open as of 31 August 2026, and glm5_next is not in the public repo's model registry. The official vLLM recipe says to use the Docker image until the integration merges. You also need FlashInfer 0.6.18 or newer for the rope-free sparse MLA path.

vllm serve zai-org/GLM-5.3-Flash \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --served-model-name zai-org/GLM-5.3-Flash

Notes that matter: FP8 KV cache is Blackwell-only — Hopper must run BF16 KV. The checkpoint carries a native MTP draft layer, so speculative decoding is free performance. Swap the model ID for zai-org/GLM-5.3-Flash-BF16 or RedHatAI/GLM-5.3-Flash-NVFP4 to change precision.

First request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3-Flash",
    "messages": [{"role": "user", "content": "Summarize sparse attention in one sentence."}],
    "temperature": 1.0,
    "max_tokens": 512,
    "chat_template_kwargs": {"reasoning_effort": "low"}
  }'

Thinking is always on. reasoning_effort accepts low, high or max, and defaults to max — which will burn a lot of tokens if you forget to set it.

How do you serve GLM-5.3-Flash with SGLang?

SGLang ships a dedicated image and a GLM-5.3-Flash cookbook with per-hardware verified recipes:

docker pull lmsysorg/sglang:glm-5.3-flash

The pattern from the verified 4× GB300 recipe:

sglang serve \
  --model-path zai-org/GLM-5.3-Flash \
  --tp-size 4 --ep-size 4 \
  --attention-backend dsa \
  --dsa-prefill-backend tilelang \
  --dsa-decode-backend tilelang \
  --linear-attn-backend triton \
  --kv-cache-dtype bfloat16 \
  --quantization fp8 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --speculative-adaptive \
  --reasoning-parser glm45 --tool-call-parser glm47 \
  --host 0.0.0.0 --port 30000

The one flag that will bite you is --mamba-full-memory-ratio. This model maintains two separate pools — a paged KV pool for the sparse-attention layers and a KDA recurrent state pool — and the KDA pool can cap max_running_requests before the KV pool is anywhere near full. The default 0.9 is a generic starting point, not a tuned one. If startup silently reduces your concurrency, that is why.

How do you run the GGUF builds with llama.cpp or Unsloth?

Upstream llama.cpp does not support GLM-5.3-Flash yet. The architecture enum on master stops at LLM_ARCH_GLM_DSA, and three competing support PRs have been open since 26 August. Every published GGUF requires a patched build — Unsloth's fork or PR #27773.

git clone --branch glm5next/upstream https://github.com/unslothai/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
  --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

pip install -U "huggingface_hub[cli]"
hf download unsloth/GLM-5.3-Flash-GGUF \
  --local-dir unsloth/GLM-5.3-Flash-GGUF --include "*UD-IQ3_XXS*"

./llama.cpp/llama-cli \
  --model unsloth/GLM-5.3-Flash-GGUF/UD-IQ3_XXS/GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf \
  --temp 1.0 --top-p 0.95 \
  --chat-template-kwargs '{"reasoning_effort":"max"}'

Set -DGGML_CUDA=OFF for CPU-only or for Apple Metal. If you would rather not build anything, Unsloth Desktop wraps the same runtime with automatic RAM offload and multi-GPU detection: unsloth run --model unsloth/GLM-5.3-Flash-GGUF:UD-IQ3_XXS.

For image and video input you also need the multimodal projector — mmproj-BF16.gguf (1.2 GB) from the same repository, loaded through llama-mtmd-cli.

Is GLM-5.3-Flash on Ollama or LM Studio yet?

Not for local weights. Ollama's library entry exposes exactly one tag — glm-5.3-flash:cloud — which routes to Ollama's hosted service rather than downloading anything. There is no local quantisation there. LM Studio has no GLM-5.3-Flash build either, which follows directly from llama.cpp not having merged support: LM Studio's engine is downstream of it.

Both will almost certainly get local support once one of the llama.cpp PRs lands. Until then, "run it in Ollama" means "call an API from Ollama's CLI."

How does this compare to running GLM-5.2 locally?

This is the comparison that makes GLM-5.3-Flash interesting. GLM-5.2 is 753B total parameters across 78 layers with 256 experts. Same 1M context, same DSA lineage — roughly 2.3× the weights.

Measured from Hugging Face repository file listings, 31 August 2026.

The punchline: the smallest possible GLM-5.2 build, a 1-bit 216.7 GB GGUF with severe quality loss, is larger than a 4-bit GLM-5.3-Flash at 199.7 GB that retains 92% of top-1 accuracy. A 512 GB Mac Studio that struggles with a heavily-degraded GLM-5.2 runs GLM-5.3-Flash at effectively full quality. If you tried GLM-5.2 locally and gave up, read our GLM-5.2 local guide for that model's constraints and then come back — Flash is the one that changes the answer.

Is running GLM-5.3-Flash locally cheaper than the API?

Almost certainly not, and the arithmetic is not close.

On OpenRouter, z-ai/glm-5.3-flash is priced at $0.075 per million input tokens and $0.25 per million output tokens. Lambda's published on-demand rate for B200 is $6.69 per GPU-hour, so the 4× B200 configuration that the vLLM recipe treats as a starting point costs $26.76/hour, or roughly $19,300 for a month of continuous running.

At $0.25 per million output tokens, that same $19,300 buys about 77 billion output tokens. To break even you would need to sustain roughly 30,000 output tokens per second, every second, for a month. No four-GPU node comes close.

So self-hosting is justified by things money does not directly measure:

• Data residency and privacy — prompts never leave your network. The usual reason regulated teams do this.
• No rate limits or provider drift — the weights you pinned are the weights you get, forever. MIT licence, no field-of-use rider.
• Latency floor — no network hop, and you control batching.
• Modification — fine-tuning, expert pruning, abliteration. Impossible through an API.
• Hardware you already own — if the GPUs are bought and idle, marginal cost is electricity.

If your reason is "the API is expensive," it isn't. If your reason is "the data can't leave," self-host. Our self-hosting LLMs guide covers the general version of this decision.

What goes wrong most often when running it?

Out of memory on load. Almost always someone sizing against 18B active instead of 320B total. Check the file size of what you downloaded and add ~10% before you blame the framework.

Concurrency lower than expected. On SGLang, the KDA state pool is capping you, not the KV pool. Raise --mamba-full-memory-ratio or pin --max-mamba-cache-size.

Sparse-MLA initialisation errors on vLLM. FlashInfer too old — you need 0.6.18 or newer.

Unsupported architecture errors. Your llama.cpp, mlx-vlm or vLLM predates glm5_next. None of the three has merged upstream support as of 31 August 2026; all working setups run a patched build or a vendor image.

Runaway token spend on reasoning. The default reasoning effort is max. Set it to low for simple queries.

Multimodal not working. Images and video need the extra components — mmproj for GGUF, torchcodec installed for video on SGLang. The vision tower is a separate 24-layer encoder, and video is capped at 240,000 visual tokens sampled at 2 FPS.

Quality collapse below 3-bit. The 1-bit build keeps 71% of top-1 accuracy. If output is subtly wrong rather than broken, that is your quantisation, not the model.

So should you run GLM-5.3-Flash locally?

A short decision rule.

If you have 128 GB or more of unified memory on a Mac, yes — this is the first frontier-class open model where a single desktop gives you a genuinely good build rather than a degraded one. Start at UD-IQ3_XXS and move up as your memory allows.

If you have a four-GPU Blackwell or Hopper node, yes — run the native FP8 checkpoint under vLLM or SGLang with MTP speculative decoding on. Long context is unusually cheap here.

If you are buying hardware specifically to save money on API calls, don't. The break-even is 30,000 output tokens per second and you will not hit it.

And if you are still on the fence, wait two weeks. The llama.cpp, mlx-vlm and vLLM support PRs are all in flight, and the difference between "build a fork" and "one command" is about to close.

FAQ

Can I run GLM-5.3-Flash locally?

Yes, if you have roughly 100 GB or more of combined RAM and VRAM. The 1-bit GGUF is 93.1 GB and runs in about 100 GB; a usable 3-bit build is 120.4 GB and wants 128 GB. The native FP8 checkpoint is 328.3 GB and needs a four-GPU server or a 512 GB Mac Studio.

How much VRAM does GLM-5.3-Flash need?

There is no single number because it depends on quantisation. The official FP8 weights are 328.3 GB (306 GiB per vLLM's recipe), so budget ~350 GB with KV cache. Four-bit is about 200 GB, three-bit about 120 GB, one-bit about 93 GB. All 320B parameters must be resident regardless of the 18B active count.

Can GLM-5.3-Flash run on a Mac?

Yes, and unified memory makes Apple Silicon unusually good at it. A 512 GB M3 Ultra runs an 8-bit MLX build (334 GB) at a measured ~22.3 tokens/sec. 192 GB machines handle 4-bit MLX at 177.5 GB; 128 GB machines handle UD-IQ3_XXS at 120.4 GB. BF16 at 642.7 GB does not fit any Mac.

Is GLM-5.3-Flash on Ollama?

Only as a cloud model. Ollama's library lists exactly one tag, glm-5.3-flash:cloud, which calls Ollama's hosted service instead of downloading weights. Local Ollama support depends on llama.cpp merging glm5_next, which had not happened as of 31 August 2026.

What quantisations are available for GLM-5.3-Flash?

The official release is native FP8 (e4m3), with an official BF16 mirror. Community builds cover NVFP4 for Blackwell, AWQ INT4, EXL3, MLX at 2–8 bit for Apple Silicon, and Unsloth Dynamic GGUFs from 1-bit (93.1 GB) to Q8_0 (341.0 GB). The GGUF and MLX paths both currently need a patched runtime.

Is running GLM-5.3-Flash locally cheaper than the API?

No, unless you already own the hardware. At $0.25 per million output tokens on OpenRouter, a month of 4× B200 rental at Lambda's $6.69/GPU-hour buys about 77 billion output tokens of API usage. Breaking even would require sustaining ~30,000 output tokens per second continuously. Self-host for privacy, control or modification — not for cost.

Which framework should I use to serve it?

vLLM or SGLang for GPU servers — both publish verified four-GPU recipes and both need their respective Docker images until upstream support merges. llama.cpp via Unsloth's fork for GGUF on Macs, CPU, or mixed CPU-GPU setups. mlx-vlm with a patched runtime for the fastest Apple Silicon path.

Does GLM-5.3-Flash support images and video locally?

Yes. It is natively multimodal with a 24-layer vision encoder handling both image and video input. On GGUF you must load the separate mmproj file; on SGLang you need torchcodec installed for video, and the processor samples at 2 FPS with a 240,000 visual-token cap.