Run Qwen3-Next-80B-A3B on Windows: 2026 Guide

Last updated: May 1, 2026.

Running the Qwen3-Next-80B-A3B model—a cutting-edge large language AI—on Windows is now achievable thanks to WSL2, NVIDIA GPU acceleration, and Docker containerization.

This guide provides a step-by-step walkthrough to install, configure, and run Qwen3-Next-80B-A3B on Windows 11, including practical examples, API usage, optimizations, and troubleshooting tips.

What changed in this 2026 refresh: Bumped vLLM to 0.20.x, FlashInfer to 0.6.x, CUDA Toolkit to 12.6, and added notes on the newer Qwen3.5 family alongside Ollama as a simpler alternative runtime.

Want the full picture? Read our continuously-updated Self-Hosting LLMs Complete Guide (2026) — hardware, ollama and vllm, cost-per-token, and when to self-host.

Want the full picture? Read our continuously-updated Qwen 3.5 complete guide — benchmarks, hardware requirements, and deployment patterns across the Qwen 3 family.

Introduction to Qwen3-Next-80B-A3B

Qwen3-Next-80B-A3B is an instruction-optimized, sparse Mixture-of-Experts (MoE) LLM developed by Alibaba AI Research. Despite having 80 billion parameters, it activates only ~3B per inference, allowing high throughput while using fewer resources. Key features include:

• Sparse MoE Architecture: 512 experts, routing ~10 per token.
• FP8 Quantization: Reduces VRAM requirements for efficient GPU usage.
• Extended Context Length: Supports 262K tokens natively, extendable to ~1M with YaRN.
• Multi-Token Prediction: Generates multiple tokens in parallel, boosting inference speed.
• Performance: Matches dense models in quality but requires less compute.

Running this model locally requires NVIDIA GPUs with CUDA support, WSL2, and proper environment setup. If you'd rather skip the MoE-specific tuning, the newer Qwen3.5-Next family (released early 2026) offers similar architecture with refreshed instruction tuning and is a near drop-in for the steps below — just swap the model ID.

Prerequisites

Hardware Requirements

• NVIDIA GPU: RTX 40/50 series or professional GPUs like RTX PRO 6000, H100, or B100. Minimum 48GB VRAM; 75GB+ recommended for FP8 weights.
• CPU: Multi-core x64 processor for smooth virtualization.
• RAM: 64GB+ recommended for model caching.

Software Requirements

• Windows 11 (latest 2026 build)
• WSL2 with Ubuntu 24.04 LTS
• NVIDIA Driver + CUDA Toolkit 12.6
• Docker Desktop (Linux containers + GPU integration)
• Python 3.11+
• vLLM Serving Framework 0.20+ (for OpenAI-compatible API)
• FlashInfer Library 0.6+ (CUDA graph optimized for LLMs)
• Qwen3-Next-80B-A3B Model Weights (FP8 quantized recommended)
• Optional: Ollama 0.5+ for a simpler one-command runtime alternative.

Benchmarking Overview

Tested on RTX 5090, CUDA 12.6, vLLM 0.20.1.

Step-by-Step Installation

Step 1: Setup WSL2 with Ubuntu

# Enable WSL and Virtual Machine Platform
wsl --install
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart

# Set default WSL version
wsl --set-default-version 2

1. Install Ubuntu 24.04 LTS from Microsoft Store.
2. Update packages:

sudo apt update && sudo apt upgrade -y
sudo apt install -y build-essential curl git python3 python3-pip

Step 2: Install NVIDIA Drivers and CUDA

1. Install NVIDIA driver for WSL.
2. Verify GPU inside WSL:

nvidia-smi

3. Install CUDA Toolkit 12.6 and PyTorch with CUDA support:

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu126

# Verify GPU
nvidia-smi

# Add NVIDIA Container Toolkit repository
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo systemctl restart docker

Step 3: Install Docker Desktop with GPU Support

1. Install Docker Desktop and enable WSL2 integration.
2. Install NVIDIA Container Toolkit inside Ubuntu:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo systemctl restart docker

3. Test GPU access in Docker:

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi

Step 4: Prepare Python Environment

pip3 install --upgrade pip
pip3 install transformers vllm>=0.20.0 flashinfer>=0.6.0

• Download or clone Qwen3-Next weights (FP8 recommended).

pip3 install --upgrade pip
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu126
pip3 install vllm flashinfer transformers

Step 5: Run Qwen3-Next-80B-A3B via Docker and vLLM

docker run -d --gpus all --name qwen3next80b -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface:rw \
  -e HF_HOME=/root/.cache/huggingface \
  vllm/vllm-openai:v0.20.1 \
  --model Qwen/Qwen3-Next-80B-A3B-Instruct \
  --host 0.0.0.0 --port 8000 \
  --async-scheduling --tensor-parallel-size=4 \
  --trust-remote-code

mkdir -p ~/models/qwen3 && cd ~/models/qwen3
git clone https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct

Notes:

• Mount Huggingface cache for efficient downloads.
• Adjust tensor-parallel-size for multiple GPUs.
• Server accessible at http://localhost:8000.

Launch the Model

1. Using vLLM Docker

docker run -d --gpus all --name qwen3next80b -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_HOME=/root/.cache/huggingface \
  vllm/vllm-openai:v0.20.1 \
  --model Qwen/Qwen3-Next-80B-A3B-Instruct \
  --host 0.0.0.0 --port 8000 \
  --async-scheduling --tensor-parallel-size=4 \
  --trust-remote-code

2. Using SGLang

pip install 'sglang[all]>=0.6.0'
sglang launch \
  --model-path ~/models/qwen3/Qwen3-Next-80B-A3B-Instruct \
  --port 8080 \
  --max-context-length 262144 \
  --speculative-algo NEXTN \
  --speculative-num-steps 3 \
  --speculative-num-draft-tokens 4

3. Using Ollama (simpler alternative)

# Ollama 0.5+ supports Qwen3-Next MoE models on WSL2 with NVIDIA GPUs
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3-next:80b-a3b-instruct-fp8
ollama serve
# Then query the OpenAI-compatible endpoint at http://localhost:11434/v1

Model Architecture

1. Sparse Mixture-of-Experts (MoE) Design

• 80B Total Parameters: Distributed across 512 experts.
• 3B Active Parameters: ~10 experts routed per token, optimizing compute.
• Router Module: Dynamically dispatches tokens to experts for efficient inference.

2. FP8 Quantization

• Weight Compression: 8-bit floating-point precision reduces VRAM by ~60%.
• FlashInfer Integration: CUDA-graph optimized for NVIDIA Blackwell/Hopper GPUs.

3. Extended Context Handling

• Native 262K Tokens: Efficient long-sequence processing.
• YaRN Rope Scaling: Extend context up to 1M tokens with minimal accuracy loss.

4. Multi-Token and Speculative Execution

• Multi-Token Prediction (MTP): Generates multiple tokens per forward pass.
• Speculative Decoding: Works with vLLM asynchronous scheduler to boost throughput.

Performance Tuning

• Adjust --tensor-parallel-size to match GPU count.
• Configure speculative decoding with vLLM --speculative-config.
• Batch multiple prompts for improved throughput.
• Preload frequent prompts to warm GPU cache.
• Enable FLASHINFER_USE_CUDA_GRAPH=1 for CUDA graph optimization.

Advanced Use Cases

1. Chained-Task Pipelines

# Extract entities
entities = generate(model, tokenizer, prompt="Extract key entities from this text: ...")

# Generate summary
summary = generate(model, tokenizer, prompt=f"Summarize these entities: {entities}")

2. Multi-Modal Input with Vision Adapter

pip3 install transformers[vision]

from transformers import AutoProcessor, AutoModelForVision2Seq
processor = AutoProcessor.from_pretrained("Your/VisionAdapter-Qwen3Next")
model = AutoModelForVision2Seq.from_pretrained("Your/VisionAdapter-Qwen3Next")

inputs = processor(images=pil_image, text="Describe this scene:", return_tensors="pt")
outputs = model.generate(**inputs)
print(processor.decode(outputs[0], skip_special_tokens=True))

Practical Examples

Example 1: Legal Contract Analysis

curl -X POST http://localhost:8000/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{ "model":"Qwen3-Next-80B-A3B-Instruct", "messages":[{"role":"user","content":"Identify risks in this contract: [contract text]"}] }'

Example 2: Code Review Assistant

from vllm import Client
client = Client("http://localhost:8000")

response = client.chat(
    model="Qwen3-Next-80B-A3B-Instruct",
    messages=[{"role":"user","content":"Review this Python function and suggest improvements:\n``````"}]
)
print(response.choices[0].message.content)

Example 3: Query via Curl

curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
  "model": "Qwen/Qwen3-Next-80B-A3B-Instruct",
  "messages": [
    {"role": "user", "content": "Explain the benefits of using WSL2 for AI model deployment."}
  ]
}'

Example 4: Query via Python Client

import requests

url = "http://localhost:8000/v1/chat/completions"
headers = {"Content-Type": "application/json"}
data = {
  "model": "Qwen/Qwen3-Next-80B-A3B-Instruct",
  "messages": [{"role": "user", "content": "Summarize advantages of sparse MoE models."}]
}

response = requests.post(url, headers=headers, json=data)
print(response.json()["choices"][0]["message"]["content"])

Run:

python3 query_qwen.py

Optimization Tips

• GPU Memory: Use FP8 weights or tensor parallelism for multi-GPU setups.
• FlashInfer: Version >=0.6.0 fixes CUDA graph errors and adds Blackwell kernels.
• CUDA & PyTorch: Use compatible versions (CUDA 12.6 + CU126 PyTorch).
• Caching: Keep Huggingface cache consistent to avoid repeated downloads.
• Verbose Logs: Enable in vLLM for debugging.
• Context Length: Modify config for extended context via YaRN scaling.

Troubleshooting

• Model fails to load: Check GPU recognition with nvidia-smi and CUDA compatibility.
• WSL2 issues: Update Linux kernel to latest stable release.
• Slow inference: Ensure FP8 quantized model is used, and async scheduling is enabled.
• Container errors: Verify Docker GPU passthrough and environment variables.

Summary

Running Qwen3-Next-80B-A3B on Windows 11 is now practical with WSL2, Docker, and NVIDIA GPU acceleration. With FP8 quantization, sparse MoE architecture, and extended context support, you can deploy large-scale, instruction-optimized AI models locally for research, NLP, multi-modal projects, and advanced chained-task pipelines. For teams that want a faster path to a working endpoint, Ollama 0.5+ provides a one-command runtime; for production-grade serving, vLLM 0.20+ remains the recommended stack in 2026.

References

1. Running Qwen3 8B on Windows: A Comprehensive Guide
2. Run Qwen 3 8B on Mac: An Installation Guide
3. Run Qwen3-8B on Ubuntu
4. Gemma 3 vs Qwen 3
5. Run Qwen3-Next-80B-A3B on macOS