Your Mac as a Local-LLM Server: What You Can Actually Run in 2026

If you already run a couple of Macs, the cheapest way to get a private, always-on inference endpoint isn't a GPU box in the closet — it's a Mac mini or Mac Studio sitting headless on your network. Apple Silicon's unified memory architecture lets the GPU address almost all of system RAM, so a 24GB Mac mini behaves a bit like a card with 24GB of VRAM, but at single-digit idle watts. This post is the server-specific angle: how big a model your RAM actually allows, how to serve it to other machines, and what realistic throughput looks like. For the full picture on Apple Silicon inference, see the pillar linked at the end.

Why is a Mac a good small inference server?

Three reasons, all downstream of unified memory:

• RAM is the constraint, not a separate VRAM pool. On a PC you're capped by your GPU's discrete VRAM (often 8–24GB). On Apple Silicon the GPU shares the machine's unified memory, so a 64GB Mac can hold models that would need a multi-GPU rig elsewhere.
• Memory bandwidth is high and the power draw is low. LLM token generation is memory-bandwidth-bound, and Apple's chips pair wide bandwidth with idle power of just a few watts. An equivalent multi-GPU build can pull several hundred watts.
• It's quiet and always-on. A near-silent mini in a cupboard, reachable over your LAN, is a genuinely pleasant home/office inference box.

The trade-off, covered below, is that you don't get the CUDA ecosystem, and batch/concurrent throughput is weaker than a real datacenter GPU. For a handful of users hitting one model, that rarely matters.

How much model can each unified-memory tier actually run?

The honest rule of thumb: budget roughly 0.5–0.7 GB per billion parameters at 4-bit for the weights, add ~15–25% on top for the KV cache, activations, and the framework, then leave a few GB for macOS itself. By default macOS lets the GPU use a large share of unified memory but not all of it, so don't plan to spend your last gigabyte. The table below maps common memory tiers to what fits comfortably — sizes are approximate and depend on quant level and context length, so treat them as a starting point, not a guarantee.

One caveat on MoE (mixture-of-experts) models: a 30B-class MoE with only a few billion active parameters is fast per token, but you still need the full quantized weight set resident in memory. The low active-parameter count helps speed, not RAM — size your memory against the total, not the active, parameter count.

A note on 2026 buying reality: a DRAM supply crunch trimmed Apple's high-memory options. Apple's static spec sheets still list higher build-to-order tiers (the M4 Max Mac Studio configurable to 64/128GB, the M3 Ultra to 256GB), but as of mid-2026 several reports describe the store quietly removing the 32GB Mac mini and the 128/256GB Mac Studio order options under supply pressure — so what's actually orderable on a given day may be narrower than the spec page suggests. The standard M4 mini ships at 16GB or 24GB, and the M4 Pro mini tops out at 48GB. Configure as much RAM as you can afford up front — it's soldered, so there's no upgrade later.

How do you serve Ollama to your network headless?

Ollama is the simplest path. The catch on macOS: when Ollama runs as the menu-bar app, it doesn't see environment variables you export in your shell, so it binds to 127.0.0.1:11434 and other machines can't reach it.

For a quick test in a terminal you control, quit the menu-bar app first, then:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

For the persistent app, Ollama's official guidance is to set the variable with launchctl and restart the app:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
# then quit and reopen Ollama

Note that launchctl setenv resets on logout. For a server that should survive reboots, put it in a login item or a small LaunchAgent plist (an EnvironmentVariables block setting OLLAMA_HOST to 0.0.0.0:11434, with RunAtLoad and KeepAlive true) instead.

From another machine, point clients at http://<mac-ip>:11434. Verify it's listening on all interfaces with lsof -iTCP:11434 -sTCP:LISTEN — you want *:11434, not localhost:11434. Two cautions:

• Treat 0.0.0.0 as LAN-only. Ollama's API has no authentication. Keep it behind your firewall; never expose port 11434 to the public internet. If you need remote access, put it behind a reverse proxy with auth or a Tailscale/WireGuard tunnel.
• macOS firewall may prompt to allow incoming connections the first time — approve it, or add the binary under System Settings → Network → Firewall.

What about LM Studio and MLX as the server?

LM Studio exposes an OpenAI-compatible API server on localhost:1234, which is handy if your apps already speak the OpenAI schema — you just change the base URL. Toggle "Serve on local network" (or run its headless daemon and CLI, lms) to bind beyond loopback, and it can keep one model pinned while others load and evict around it. Unlike Ollama, LM Studio supports API tokens — they're off by default, but you can enable authentication and create tokens in Server Settings, which you should do before binding beyond localhost. Even so, keep it on the LAN.

If you want to serve MLX directly without a GUI, Apple's mlx-lm ships its own OpenAI-compatible server:

pip install mlx-lm
mlx_lm.server --model mlx-community/Qwen2.5-7B-Instruct-4bit --host 0.0.0.0 --port 8080

Then point clients at http://<mac-ip>:8080/v1. The mlx-lm docs note its built-in server is meant for local/dev use and has only basic security checks, so the same rule applies — LAN-only, behind a tunnel if it needs to leave the network.

A 2026 wrinkle worth knowing: Ollama itself now uses MLX on Apple Silicon. Since the 0.19 release (March 2026), Ollama runs supported safetensor/MLX-format models through Apple's MLX framework by default on Apple Silicon and falls back to llama.cpp/GGUF for broad compatibility. So the old "Ollama is llama.cpp, LM Studio is MLX" split no longer holds — which backend you get now depends on the model format more than the tool. LM Studio and mlx_lm.server still matter when you want explicit MLX workflows or GUI model management.

What tokens/sec should you realistically expect?

Throughput scales with memory bandwidth and shrinks as the model grows. These are approximate single-stream generation figures gathered from public 2026 benchmarks — treat them as ballpark, not guarantees, since quant level, context length, prompt size, and backend all move the number:

• 7B–8B at Q4: roughly 75–90 tok/s on an M4 Max with an MLX backend; mid-tier chips (M4 Pro) land lower, often ~30–40 tok/s.
• 30B-class MoE (few active params): roughly 60–90 tok/s on an M4 Max — MoE punches above its weight here.
• 70B dense at Q4: roughly 15–22 tok/s on an M4 Max.

On the framework question: on supported Apple Silicon models, MLX-backed runtimes can be materially faster than llama.cpp on Metal — public tests put the gain anywhere from ~20% to over 40% on small-to-mid models, and Ollama's own MLX preview reported a large jump on a 35B-A3B MoE. But the exact delta depends on model format, quantization, prompt length, and cache behaviour, and it narrows on larger models (27B+) where both paths become memory-bandwidth-bound and converge. Don't bank on a single universal percentage.

For context, faster than reading speed is ~7–10 tok/s, so even a 70B model on a well-specced Mac is comfortably usable interactively. Where Macs fall behind a GPU is concurrency: serving many simultaneous requests, the per-request speed drops faster than on a batching-optimised GPU. For one to a few users, you won't notice.

Mac mini or Mac Studio as the server?

Pick by the model size you actually need resident:

• Mac mini (M4 / M4 Pro) — the value pick. The base M4 at 24GB happily serves up to ~14B models; the M4 Pro at 48GB handles 32B-class comfortably and dabbles in 70B at low quant. Idle draw is a few watts; under inference it usually sits well below Apple's published system maximum (65W for the M4 mini, 140W for the M4 Pro mini). This is the right "home/office endpoint for one team" box.
• Mac Studio (M4 Max / M3 Ultra) — the capacity pick. You're paying for more memory and more bandwidth, which both raises the model ceiling and speeds large-model generation. Apple's published system max is 145W for the M4 Max Studio and 270W for the M3 Ultra — still modest for the work. Choose it when you need 70B at decent quant with long context, or want several large models resident at once.

Rule of thumb: if your target model fits in 48GB, a Mac mini M4 Pro is the better dollar-per-token-served. Above that, the Studio earns its premium.

What are the limits of a Mac inference server?

• No CUDA ecosystem. Tools, kernels, and quant formats that assume NVIDIA won't all work. You live in the llama.cpp/MLX world — broad and good, but not everything.
• Batch/throughput ceiling. Great for low-concurrency interactive use; a single GPU with proper batching will out-serve it under heavy parallel load.
• Soldered RAM. No upgrades. Buy the memory you'll want in two years, today.
• Prompt-processing (prefill) can lag generation on very long contexts, since prefill leans more on compute than bandwidth.

None of these disqualify a Mac as a small server — they just mean it's the wrong tool for a high-QPS production API and the right tool for a private team endpoint.

Frequently asked questions

Can a Mac mini run a 70B model as a server?

Only the higher-memory configs, and only at aggressive quantization. A 48GB M4 Pro mini can load a 70B model at low quant (Q3/Q4) with short context, but it's tight; 64GB+ is where 70B becomes comfortable, which on current hardware means stepping up to a Mac Studio. For most teams a 32B-class model on a 48GB mini is the better balance of quality and headroom.

Is Ollama or LM Studio better for serving on a Mac?

Ollama is simpler to run headless and script, and as of 0.19 it uses MLX on Apple Silicon for supported models. LM Studio gives you an OpenAI-compatible endpoint, GUI model management, and optional API-token auth. Use Ollama for simplicity; reach for LM Studio (or mlx_lm.server) when you want explicit MLX control or token-gated access.

Do I need to expose port 11434 to the internet?

No — and you shouldn't. The Ollama API has no built-in authentication, and LM Studio's is off by default. Bind to 0.0.0.0 for LAN access only, and if you need it off-network, use a VPN/Tailscale tunnel or an authenticating reverse proxy. A public, unauthenticated inference endpoint is an open invitation.

How much faster is MLX than the llama.cpp backend?

On supported Apple Silicon models, public 2026 tests put MLX anywhere from roughly 20% to over 40% faster for token generation on small-to-mid models. The gap narrows on larger models (27B+) where both backends become memory-bandwidth-bound and converge. Note that Ollama now uses MLX by default on Apple Silicon, so the comparison is increasingly about model format, not which tool you picked.

How much electricity does a Mac LLM server use?

Very little. Apple Silicon Macs idle at a few watts, and under inference they typically run below Apple's published system maximums — which range from 65W for the M4 mini up to 270W for the M3 Ultra Studio. That's far below a comparable multi-GPU build, and annual running cost is usually in the low tens of dollars.

What's the single most important spec to choose?

Unified memory. It sets the hard ceiling on model size, it can't be upgraded later, and it's the spec most likely to bottleneck you. Bandwidth (which tracks the chip tier) sets speed; RAM sets what's even possible. Buy memory first.

Setting up a private inference box and want a second pair of hands on the integration — wiring it into your apps, internal tools, or CI? Codersera can help you extend your team with vetted remote developers who've shipped this kind of work.