` - Always run `pnpm lint` and `pnpm test` before committing. ``` ### Example 2: GitHub's spec-kit (real, large) The `github/spec-kit` `AGENTS.md` documents an "integration architecture" for adding new AI agents to a CLI toolkit. Sections include *About Spec Kit and Specify*, *Integration Architecture*, *Adding a New Integration* (a seven-step playbook), *Command File Formats* (Markdown vs TOML vs YAML), *Argument Patterns* (`$ARGUMENTS` vs `{{args}}` vs `{SCRIPT}`), *Branch Naming Convention*, and *Common Pitfalls*. The file is dense, opinionated, and treats the agent like a teammate who needs to know the team's conventions on day one. ### Example 3: Node monorepo with scoped boundaries Factory's documentation shows a Node monorepo with this shape: ``` # Acme Platform ## Stack - TypeScript 5.4, pnpm 9, Turbo 2, Vite 5, Postgres 16. ## Layout - packages/api → Express server (only edit code here for API changes) - packages/web → React + Vite (only edit code here for UI changes) - packages/shared → cross-cutting types and zod schemas ## Commands - Install: `pnpm install` - Dev: `pnpm dev` - Test: `pnpm test --filter ` - Lint: `pnpm lint` - Build: `pnpm build` ## Boundaries - Never edit `prisma/migrations/`; generate via `pnpm db:migrate`. - Never commit `.env*` files. - Ask before adding a new top-level dependency. ## PRs - Conventional commits. - Squash on merge. - Include a `Testing` section in the PR body. ``` Note the *Layout* section's "only edit code here for X" — this is one of the highest-leverage patterns. A single sentence per package eliminates the most common monorepo agent failure: cross-package edits that shouldn't exist. ### Example 4: Polyglot repo with language-specific blocks ``` # DataPlatform ## Languages This repo is Python + Go + TypeScript. Each top-level service is single-language. ### Python services - Format: `ruff format`, then `ruff check --fix`. - Type-check: `mypy --strict`. - Test: `pytest -x`. ### Go services - Format: `gofumpt -w`. - Vet: `go vet ./...`. - Test: `go test -race ./...`. ### TypeScript services - Format: `biome format --write`. - Lint: `biome check`. - Test: `vitest run`. ## Boundaries - Never call a Go service from a Python service directly; always go through the gRPC contracts in `protos/`. - Never edit generated protobuf code in `gen/`. Regenerate via `buf generate`. ``` ### Example 5: Framework-specific (Next.js) ``` # MyShop ## Stack Next.js 16 (Pages Router for marketing, App Router for /shop), Tailwind v4 in /shop only, styled-components in marketing only. ## Setup - Node 22.x, pnpm 9. - `pnpm install` - `pnpm dev` → http://localhost:3000 ## Test - `pnpm test` → vitest unit - `pnpm e2e` → Playwright - `pnpm lint:check` → ESLint - `pnpm type:check` → tsc --noEmit ## Conventions - Pages Router files: kebab-case (`product-detail.tsx`). - App Router routes: lowercase, no underscores. - Server components by default; mark client components with the `'use client'` directive at the top. - Tailwind classes only in `app/shop/**`; styled-components only in `pages/**`. ## Don't - Don't mix Tailwind and styled-components in the same file. - Don't add new Pages Router routes; new routes go in App Router under `app/shop/`. - Don't commit `.env.local` or anything in `secrets/`. ``` ## Patterns that actually change agent behaviour The reason `AGENTS.md` works is not because LLMs are great at following instructions — they aren't. It's that certain phrasings dramatically improve adherence. Patterns we've seen consistently move the needle: - **Imperative voice, not descriptive.** "Run `pnpm test` before every commit" works better than "this project uses pnpm test for testing." - **Three-tier boundaries.** Group rules into *always do*, *ask first*, and *never do*. The explicit hierarchy gives the agent a clean classification problem instead of an open-ended judgement call. - **Concrete commands with flags.** "Run `pytest -x --tb=short tests/unit`" beats "run the unit tests." - **Tech stack with version numbers.** "Node 22.x, pnpm 9, Vite 5" beats "modern JS stack." - **Real code examples.** A 5-line example of your preferred error-handling pattern is worth a paragraph of prose. - **Scope sentences per directory.** "Backend code only in `packages/api/src/`" eliminates a class of failures by itself. - **Generated-file guards.** List every directory that is regenerated by a build step (`gen/`, `__generated__/`, `migrations/`, `dist/`) and tell the agent never to edit them. ## Anti-patterns: ten things not to do 1. **Don't write a persona.** "You are a helpful AI assistant who loves clean code" wastes tokens and changes nothing. The agent already knows it is an agent. 2. **Don't paste your README.** Duplicating `README.md` content in `AGENTS.md` doubles maintenance and signals nothing new to the agent. 3. **Don't include marketing copy.** "Acme Platform is the leading developer-first observability solution" is text the agent does not need. 4. **Don't write essays.** If a section is over ~30 lines, it is probably reference documentation that belongs in `docs/` and should be linked, not inlined. 5. **Don't restate language defaults.** "Use Python 3" is implicit from `pyproject.toml`. "Use Python 3.12 with the `uv` package manager, not pip" is useful. 6. **Don't conflict with `.cursorrules` / `CLAUDE.md`.** If you keep more than one rules file, treat `AGENTS.md` as the source of truth and have the others `@import` it. 7. **Don't include secrets.** Real API keys, real DB connection strings, real internal URLs. The file is in version control; assume it leaks. 8. **Don't be vague about "good code".** "Write clean, idiomatic code" is signal-free. "Use `Result<T, E>` for fallible operations, never raise unchecked exceptions" is signal. 9. **Don't ship a 2000-line file.** Codex caps at 32 KiB. Most agents start ignoring after the first few thousand tokens. Keep it under ~500 lines; push the rest into nested per-package files or `docs/`. 10. **Don't forget to update it.** Stale `AGENTS.md` is worse than no `AGENTS.md` — the agent will confidently follow rules that no longer apply. Treat it like the README: update in the same PR as the change it documents. **Building an AGENTS.md for a Cursor-heavy team?** Read our [**Cursor IDE complete guide**](https://codersera.com/blog/cursor-ide-complete-guide-2026/) for the deeper story on how Cursor parses `AGENTS.md` alongside `.cursor/rules/*.mdc`, when path-scoped MDC rules beat a global AGENTS.md, and which Project Rules to set for a 10-engineer team. ## Monorepo patterns: nested AGENTS.md files The single biggest scale lever in the spec is nested AGENTS.md. You can put one at the repo root *and* one inside `packages/api/` *and* one inside `packages/web/`. Agents walk up the directory tree from the file they're editing and combine every `AGENTS.md` they encounter, with the closest one winning on conflicts. OpenAI's own Codex monorepo ships **88** nested `AGENTS.md` files — one per service or package. The shape is: ``` repo/ ├── AGENTS.md # global: stack, root commands, branch naming ├── packages/ │ ├── api/ │ │ ├── AGENTS.md # api: Express, Postgres conventions, test command │ │ └── src/... │ ├── web/ │ │ ├── AGENTS.md # web: React, Tailwind, Vite, component conventions │ │ └── src/... │ └── shared/ │ ├── AGENTS.md # shared: zod schemas, no DB imports allowed │ └── src/... └── tools/ └── etl/ ├── AGENTS.md # etl: Python, pandas, sqlmodel; different lint rules └── ... ``` A few practical rules: - **Defaults at the root, overrides at the leaf.** Put the stack, the branch naming, the security boundaries at the root. Put the per-package test command and the per-package conventions at the leaf. - **Don't re-state inheritance.** If the root says "pnpm 9," the leaf shouldn't say "pnpm 9" again. The agent already has the root context. - **Codex-only escape hatch:** `AGENTS.override.md` at any level *replaces* rather than extends parent instructions. Use sparingly; other tools don't honour it. - **Keep nested files short.** 30–80 lines each is the sweet spot. The root file can be longer (~200 lines). ## SKILL.md: a complementary, not competing, standard SKILL.md is Anthropic's separate spec for packaging reusable agent *capabilities*. It is more structured than AGENTS.md, with mandatory YAML frontmatter, a defined directory layout, and a three-level loading model (metadata always loaded, body loaded on trigger, bundled scripts and references loaded on demand). ### The shape of a SKILL.md ``` my-skill/ ├── SKILL.md # required: frontmatter + instructions ├── scripts/ # optional: executable code Claude runs via bash │ └── validate.py └── references/ # optional: docs loaded only when explicitly referenced └── schema.json ``` The SKILL.md itself: ``` --- name: pdf-processing description: Extract text and tables from PDF files, fill forms, merge documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction. --- # PDF Processing ## Quick start Use pdfplumber to extract text from PDFs: ```python import pdfplumber with pdfplumber.open("document.pdf") as pdf: text = pdf.pages[0].extract_text() ``` For advanced form filling, see [FORMS.md](FORMS.md). ``` ### Required frontmatter fields - `name` — lowercase, hyphens-only, max 64 characters, cannot contain "anthropic" or "claude" or XML tags. - `description` — non-empty, max 1024 characters, no XML tags. This is the *primary triggering signal*; Anthropic explicitly advises making it "pushy" about use cases to combat under-triggering. Optional: `license`. The spec is published in the open at [anthropics/skills](https://github.com/anthropics/skills) and at `agentskills.io`. ### When to use SKILL.md vs AGENTS.md | You want to… | Use | | --- | --- | | Tell every agent how to build, test, and not-break *this repo* | `AGENTS.md` | | Package a reusable capability (PDF processing, Excel generation, internal API client) with bundled scripts | `SKILL.md` | | Share project conventions across tools (Codex, Cursor, Copilot, Aider…) | `AGENTS.md` | | Ship a capability that loads only when triggered, to save context | `SKILL.md` | | Add per-package rules in a monorepo | Nested `AGENTS.md` | | Distribute capabilities to a Claude workspace, claude.ai, or Claude Code installs | `SKILL.md` | The two formats overlap in spirit (both are filesystem-resident instructions for AI agents) but solve different problems. `AGENTS.md` answers "how does this repo work?" `SKILL.md` answers "how does this reusable capability work, and when should you trigger it?" A serious team will use both: `AGENTS.md` at the repo root, and one or more `SKILL.md` bundles in `.claude/skills/` (Claude Code) or uploaded via the API. ## Migrating from .cursorrules to AGENTS.md `.cursorrules` is the legacy Cursor format. As of 2026 Cursor still reads it, but the official guidance is to migrate. The mechanical conversion is straightforward: 1. **Audit what's in `.cursorrules`.** Most files are 80% universal ("use TypeScript, format with Prettier, prefer functional components") and 20% Cursor-specific ("in Agent mode, ask before deleting files"). 2. **Move the universal 80% verbatim to `AGENTS.md`.** Same Markdown, same headings — just rename and relocate. 3. **Keep Cursor-specific rules in `.cursor/rules/*.mdc`.** The MDC format supports YAML frontmatter (`description`, `globs`, `alwaysApply`) for path-scoped rules. Use MDC for "when editing `**/migrations/*.py`, always include a down-migration" type rules; use AGENTS.md for "use Postgres parameterised queries" type rules. 4. **Delete `.cursorrules` once `AGENTS.md` is live**, or keep it as a thin pointer that says "see AGENTS.md." 5. **If you also use Claude Code,** create a one-line `CLAUDE.md` that opens with `@AGENTS.md` to import the file into Claude's context. Teams that complete this migration end up with one source of truth (`AGENTS.md`), one tool-specific surgical override location per tool, and zero drift between formats. **Comparing Claude Code, Codex, and Cursor head-to-head?** Our [**Claude Code vs OpenAI Codex deep dive**](https://codersera.com/blog/claude-code-vs-openai-codex-2026/) walks through how each one reads `AGENTS.md` in practice, where their fallback behaviours differ, and which to pick for an extended remote team. ## AGENTS.md and MCP: how they fit together A common confusion: does `AGENTS.md` replace MCP (Model Context Protocol) server configuration? No — they sit at different layers. - **`AGENTS.md`** is project-level *instructions*: what to do, what not to do, how to test, how to commit. - **MCP servers** are project-level *tools*: an HTTP/stdio endpoint the agent can call to read your database, query your monitoring system, fetch a Jira ticket, or run a custom script. A well-configured repo has both. `AGENTS.md` at the root tells the agent "run the test command before committing." A `.mcp.json` (Claude Code, Codex) or `.cursor/mcp.json` (Cursor) tells the agent *how* to call your in-house MCP servers. The instructions in `AGENTS.md` can reference MCP tools by name ("use the `db-readonly` MCP server for schema questions, never query the prod DB directly"). ## Rollout checklist for a 10-engineer team 1. **Day 1.** Audit every existing tool-specific rules file (`.cursorrules`, `.continuerules`, `.github/copilot-instructions.md`, `CLAUDE.md`). Note what's universal vs tool-specific. 2. **Day 1.** Write a 100–200 line `AGENTS.md` at the repo root covering the 8 recommended sections (overview, setup, test, style, structure, git, security, boundaries). 3. **Day 1.** Have one engineer who uses each tool (Cursor, Claude Code, Codex, Copilot) run a real task with the new file and report drift. Iterate. 4. **Day 2.** Add `@AGENTS.md` at the top of `CLAUDE.md`, plus any Claude-Code-specific subagent / slash-command guidance. 5. **Day 2.** Move Cursor-specific path-scoped rules into `.cursor/rules/*.mdc`; delete or thin `.cursorrules`. 6. **Week 1.** Add nested `AGENTS.md` files for the top 3 packages in the monorepo. 7. **Week 2.** Wire `AGENTS.md` updates into the PR template. Any structural change (new test command, new lint rule, new boundary) requires an `AGENTS.md` edit in the same PR. 8. **Month 1.** Review the rate of agent rule-violations in code review. Common patterns of violation are signals for new `AGENTS.md` sections. ## FAQ ### Is AGENTS.md officially a standard? Yes — it is stewarded by the Agentic AI Foundation under the Linux Foundation, which gives it vendor-neutral governance. The format originated from collaboration between OpenAI, Amp (Sourcegraph), Google (Jules), Cursor, and Factory, and is now adopted by more than 60,000 public repositories. ### Does Claude Code read AGENTS.md natively? Not directly. Claude Code reads `CLAUDE.md` by convention. The recommended workaround is to put `@AGENTS.md` as the first line of `CLAUDE.md` — Claude Code's `@`-import syntax pulls the contents of `AGENTS.md` into its context. Some installs also recognise the singular `AGENT.md` filename. ### Should I write AGENTS.md or .cursorrules? Write `AGENTS.md`. `.cursorrules` is Cursor-only and officially in deprecation. Anything you'd put in `.cursorrules` works in `AGENTS.md` and is portable to every other tool. ### How long should AGENTS.md be? Aim for 100–300 lines at the repo root. Codex caps at 32 KiB; other tools start ignoring after the first few thousand tokens. For larger projects, split into nested `AGENTS.md` per package rather than letting the root file grow past ~500 lines. ### Do I need YAML frontmatter? No. The 1.0 spec defines none. The 1.1 proposal (open, not merged) adds optional `description` and `tags` fields. Including them is forward-compatible and costs nothing; depending on them for correctness is not safe yet. ### AGENT.md or AGENTS.md? Always uppercase. Plural is the canonical form (`AGENTS.md`); singular (`AGENT.md`) is a fallback that some tools (Kilo, certain Claude Code installs) accept. Don't ship lowercase — case-sensitive filesystems treat `agents.md` as a different file. ### Can I put secrets in AGENTS.md? Never. The file is committed to version control. Treat it as fully public — anything you wouldn't put in `README.md` should not be in `AGENTS.md`. For secrets, document the *names* of the env vars and where to fetch their values from (Vault, AWS Secrets Manager, 1Password), not the values themselves. ### How does the monorepo precedence work exactly? Tools walk from the file being edited up to the repo root and load every `AGENTS.md` they pass. The nearest file's instructions take precedence on conflict. Codex concatenates root-down; Factory walks nearest-first; Cursor combines them with explicit MDC rules winning. The differences rarely matter as long as your nested files don't actively contradict the root. ### Is SKILL.md the same as AGENTS.md? No. `AGENTS.md` is project-level conventions read by 20+ coding agents. `SKILL.md` is Anthropic's separate spec for packaging reusable capabilities (with required YAML frontmatter, scripts, references, and a progressive-disclosure loading model) read by Claude products. Use both: `AGENTS.md` for repo conventions, `SKILL.md` for reusable capability bundles. ### Does AGENTS.md affect cost? Yes, mildly. A 200-line file is ~1–2k tokens, loaded on every agent task. The savings (fewer wrong-direction edits, fewer test-command retries, fewer "what's the lint command?" exchanges) more than offset the load cost in any non-trivial repo. Don't bloat the file past usefulness, but don't optimise for token-thrift either. ### What if my agent ignores AGENTS.md? First, check that the filename is `AGENTS.md` (uppercase, plural) at the actual working directory. Second, check that the file is under 32 KiB. Third, restart the agent session — most tools load the file on session start, not on every prompt. Fourth, check tool docs for known fallbacks (Claude Code reads `CLAUDE.md`; Gemini CLI reads both; Cursor needs MDC rules to be enabled in settings). ### Should our public OSS repo have AGENTS.md? Yes. External contributors increasingly use agents to draft PRs against your repo. An `AGENTS.md` at the root sharply improves the quality of agent-generated contributions, reduces CI cycle waste, and gives human contributors a parallel reading of "how this project actually works." It is now table-stakes for any public repo expecting external contribution. ## What this means for engineering leadership The bar for "AI-ready engineering team" has moved up. Two years ago, having any rules file at all was enough. Today, the teams getting the most out of coding agents have invested in a layered structure: a universal `AGENTS.md` at the root, nested `AGENTS.md` per package in monorepos, tool-specific overrides for the agents they standardise on, MCP server configuration alongside, and a process discipline that updates `AGENTS.md` in the same PR as the change it documents. For hiring, the signal we look for at Codersera is exactly this orchestration fluency. The engineers who deliver in 2026 don't write the most code by hand — they write the rules files, design the MCP integrations, scope the agent runs, and review the diffs critically. When we vet engineers for client teams, comfort with `AGENTS.md` authoring, monorepo nesting strategy, and the AGENTS-vs-SKILL.md-vs-MCP decomposition is now part of the technical-fit conversation, not a "nice to have." ## References and further reading - [agents.md — official spec page](https://agents.md/) - [agentsmd/agents.md — Linux Foundation repo](https://github.com/agentsmd/agents.md) - [AGENTS.md v1.1 proposal (Issue #135)](https://github.com/agentsmd/agents.md/issues/135) - [OpenAI Codex — Custom instructions with AGENTS.md](https://developers.openai.com/codex/guides/agents-md) - [Factory CLI — AGENTS.md configuration](https://docs.factory.ai/cli/configuration/agents-md) - [Kilo Code — AGENTS.md docs](https://kilo.ai/docs/customize/agents-md) - [GitHub Blog — How to write a great AGENTS.md (lessons from 2,500+ repos)](https://github.blog/ai-and-ml/github-copilot/how-to-write-a-great-agents-md-lessons-from-over-2500-repositories/) - [anthropics/skills — Anthropic SKILL.md reference repo](https://github.com/anthropics/skills) - [Anthropic — Agent Skills overview](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) - [github/spec-kit — real-world AGENTS.md example](https://github.com/github/spec-kit/blob/main/AGENTS.md) - [Codersera — Cursor IDE complete guide 2026](https://codersera.com/blog/cursor-ide-complete-guide-2026/) - [Codersera — AI coding agents complete guide 2026](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) - [Codersera — Claude Code vs OpenAI Codex (2026)](https://codersera.com/blog/claude-code-vs-openai-codex-2026/) - [Codersera — Best MCP servers for Claude Code and Cursor](https://codersera.com/blog/best-mcp-servers-claude-code-cursor-2026/) - [Codersera — Grok Build vs Claude Code vs Codex CLI](https://codersera.com/blog/grok-build-vs-claude-code-vs-codex-cli-2026/) ## Next steps If you're scaling an engineering team in 2026, the gap between "team with a working `AGENTS.md` playbook" and "team without one" shows up in code-review velocity, agent-failure rate, and onboarding time within a single sprint. Get the foundation right, and the rest of the AI-coding-agent stack (Cursor, Claude Code, Codex, Copilot, MCP, SKILL.md) compounds on top of it. If you're extending your engineering team and want senior, vetted, remote engineers who already work this way in production — engineers fluent in `AGENTS.md` authoring, monorepo nesting, MCP integration, and the agent-orchestration discipline this guide covers: [Hire a Codersera-vetted senior engineer](https://codersera.com/hire). Risk-free trial, faster hiring, lower hiring risk, and engineers who can extend your team without extending your context-switching cost. =============================================================== [12/20] Cursor IDE in 2026: The Complete Developer's Guide URL: https://codersera.com/blog/cursor-ide-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer (refreshed May 2026).** Cursor 3.5 launched on May 20, 2026 with the headline feature of **Cloud Agents** — agents that run in isolated cloud VMs with full terminal, browser, and desktop access, can work across multiple repos in parallel, and report results back to your IDE asynchronously. Cursor 3.3 (May 7) added **Build in Parallel** subagents, pinned-skill pills, Composer 2.5 with multi-file refactor at file-tree scale, and native Jira integration. Pricing tiers are unchanged from 3.2; Cloud Agents are metered separately. The competition picture has also shifted — Claude Code 2.1 with Opus 4.7 hit 80.8% on SWE-bench Verified (the highest in the category, April 2026), Cursor 3.5 is in the same ballpark for in-IDE work, and the right choice now depends more on your workflow (terminal-first vs IDE-first vs cloud-async) than on raw quality. The rest of this guide covers Cursor's core agent, Composer, model picker, team workflows, pricing, and a comparison vs Claude Code and GitHub Copilot agent mode. The May 2026 updates are summarised in the section that immediately follows. ## What changed in Cursor between March and May 2026? Three releases in eight weeks moved the product meaningfully: - **Cursor 3.3 (May 7, 2026)** — Build in Parallel: spawn N subagents from one prompt, each tackling a different sub-task, results merged. Pinned-skill pills surface frequently-used custom skills directly in the composer bar. Composer 2.5 ships multi-file refactor at file-tree scale (closer to Claude Code's batch-edit pattern). Native Jira integration: link tickets, draft PRs that auto-update issue status. - **Cursor 3.5 (May 20, 2026)** — Cloud Agents. Spin up an isolated VM, hand it a task, it runs terminal commands, browses, and edits files autonomously, with you tracking progress async in the IDE. Pinned to specific repos or repo-groups. Metered pricing on top of your existing seat. - **Plug-ins and the wider agent ecosystem** — Cursor's MCP server support, which landed late 2025, has matured; the public registry of community MCP servers crossed 200 entries by May 2026. OpenCode (the open-source coding-agent project) hit 161K GitHub stars and added a "Scout" subagent + auto-compact — worth knowing about even if you stay on Cursor. For the comparison against Claude Code, GitHub Copilot agent mode, OpenCode, and Cline as of May 2026, see our [AI coding agents complete guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/). For the rest of the Cursor surface — Composer, agent mode, model picker, pricing, team policies — read on. Cursor went from "AI-fork of VS Code" to the default editor inside many engineering orgs in roughly eighteen months. By mid-2026 it's being used by more than a million developers and ~360,000 paying customers, and it sits inside 64% of the Fortune 500. Whether you're a solo developer evaluating it for the first time, a tech lead negotiating a Teams seat count, or a CTO standardizing tooling across an extended team, the choices Cursor exposes (modes, models, rules, indexing, MCP, privacy) directly affect how your engineers ship. This guide is the version we wish we'd had when we started running Cursor in production with our own engineers and the engineers we place at client teams. It's opinionated, version-stamped, and grounded in what Cursor actually behaves like in real codebases — not just what the docs say. *Last updated: May 1, 2026.* ## TL;DR - **What it is:** Cursor is an AI-native IDE forked from VS Code, with deeply integrated agent, ask, and edit modes, codebase indexing, MCP tool support, and a model router that fronts Claude, GPT, Gemini, and DeepSeek frontier models. - **Pricing:** Six tiers — Hobby (free), Pro ($20), Pro+ ($60), Ultra ($200), Teams ($40/user), Enterprise (custom). Mid-2025 the model shifted from request quotas to a credit-based usage pool. - **Workflow:** Plan in Ask mode, implement in Agent mode, fall back to Manual edit when you want surgical control. This pattern outperforms "just talk to the agent" by a wide margin. - **Models that matter in 2026:** Claude Opus 4.7 (architecture, refactors), GPT-5.5 (general-purpose coding), DeepSeek V4 Pro (cheap reasoning), plus any OpenAI-compatible custom endpoint via your own key. - **Where it breaks:** Long agent sessions degrade reasoning quality; monorepos without `.cursorignore` hammer your indexer; multi-file refactors produce messy PRs that need human review. - **Codersera's take:** Cursor is the right default for most teams, but treat the agent as a junior engineer who needs guardrails, reviews, and a tightly scoped working set — not as an autonomous senior. ## Cursor 3.0 / 3.2: the Agents Window era Cursor 3.0 shipped on April 2, 2026 and structurally changed what "Cursor" means. The [3.0 changelog](https://cursor.com/changelog/3-0) introduces an **Agents Window** — a dedicated surface (open with `Cmd+Shift+P → Agents Window`) that "allows you to run many agents in parallel across repos and environments: locally, in worktrees, in the cloud, and on remote SSH." The IDE is still there, but for agent-heavy workflows it has been demoted to one of several panes. Coverage like [The New Stack's "Cursor's $2 billion bet: the IDE is now a fallback, not the default"](https://thenewstack.io/cursor-3-demotes-ide/) captures the framing the team is leaning into. The headline new commands are `/worktree` and `/best-of-n`, both documented in [Cursor's worktrees guide](https://cursor.com/docs/configuration/worktrees). `/worktree` tells Cursor to "do the rest of that chat in a separate checkout," so the agent's edits live in an isolated git worktree instead of mutating your main checkout. `/best-of-n` "runs the same task across multiple models at once. Each run gets its own worktree, so the candidates stay isolated from each other and from your main checkout." A real example from the docs: ``` /best-of-n sonnet,gpt,composer fix the flaky logout test ``` Three agents race the same task in three isolated worktrees; you compare diffs, pick the winner, discard the rest. Combined with the **Agent Tabs** view (multiple chats in side-by-side or grid layouts) and **Design Mode** (annotate browser UI directly to feed visual feedback back into an agent), the practical workflow shifts from "talk to one agent in one chat" to "dispatch 4–8 agents on parallel branches and adjudicate." If you used the Cursor 1.x background-agents and 2.x multitask-style flows, 3.0 is the version where this stopped being a novelty and became the primary surface. The 3.2 point release continued this trajectory with faster large-file diffs, an `Await` tool for monitoring background runs, and screenshot-based browser-automation clicking. ## 1. What Cursor actually is, and why it took off Cursor is a desktop IDE built on a fork of VS Code, with AI as a first-class primitive rather than a sidebar extension. You get the entire VS Code extension ecosystem — Prettier, ESLint, Docker, GitLens, language servers — and on top of that, three AI interaction modes (Ask, Agent, Manual), inline tab-completion, codebase-aware embeddings, MCP tool integrations, and project-scoped rules. The reason it stuck where Copilot didn't is simple: Cursor treats your repository as the unit of context, not the cursor position. Tab completion uses a fast model trained on your edits; chat and agent calls run through frontier models; both pull in semantically relevant chunks of your codebase via a vector index. The result is that the AI knows about the function three files away, not just the line you're editing. That UX advantage compounded as frontier models got better. The same Cursor session that produced mediocre output on GPT-4 in 2024 now produces production-grade refactors on Claude Opus 4.7 in 2026 — same product, different ceiling. We've covered the model side of this in [how to use Claude 4 and Sonnet with Cursor and Windsurf](https://codersera.com/blog/how-to-use-claude-4-and-sonnet-with-cursor-and-windsurf/), and the broader model landscape in our [DeepSeek V4 complete guide for 2026](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). ## 2. The three-mode workflow: Ask, Agent, Manual Cursor's interaction model is the single biggest thing engineers get wrong on day one. There are three modes, switched with `Cmd+.` or the dropdown in the chat panel, and each is right for a different kind of task. ### Ask mode Ask is read-only. It searches your codebase, answers questions, explains code, drafts plans — but it never writes a file. Use Ask to explore unfamiliar territory ("how does authentication flow through this service?"), to argue out an approach before committing to it, or to get a second opinion on a design. The official Cursor docs explicitly recommend planning in Ask before implementing in Agent, and the difference shows up in output quality. ### Agent mode Agent is the autonomous mode. You describe a task; Cursor reads files, edits multiple files, runs terminal commands, hits the web if it needs to, and iterates on errors. This is where the magic happens — and also where bad runs go off the rails. Agent mode rewards tightly scoped tasks with clear acceptance criteria ("add a rate-limit middleware to the /api/ingest route, write a Vitest test, run the tests"). It punishes vague instructions ("clean up the auth code") with sprawling, hard-to-review diffs. ### Manual edit mode Manual (formerly known as Composer) is for surgical, multi-file edits where you want the model to propose changes but not execute commands or wander off-task. It's the sweet spot when you know what you want changed and just don't want to type it. Older tutorials referencing "Composer" map to this mode. The pattern that consistently produces the best results: **scope in Ask, plan in Ask, implement in Agent, polish in Manual**. Skipping Ask is the most common cause of "Cursor wrote me garbage" complaints we hear from engineers we onboard. ## 3. Pricing in 2026 Cursor moved from request-based quotas to a credit-based pool in mid-2025. Each frontier-model call deducts from your monthly credit balance based on the model and request size. Heavy Opus 4.7 users burn credits faster than DeepSeek V4 users for the same task; the new system rewards model-routing discipline. | Plan | Price (USD/mo) | Best for | Key entitlements | | --- | --- | --- | --- | | Hobby | Free | Evaluation, side projects | Limited completions and agent requests; no MCP credits | | Pro | $20 (or $16 annual) | Individual professionals | $20 credit pool, frontier models, MCP, cloud agents, unlimited Tab | | Pro+ | $60 | Heavy daily users | 3x usage credits over Pro | | Ultra | $200 | Power users, indie founders | 20x credits across OpenAI/Anthropic/Google models, priority access to new features | | Teams | $40/user | Engineering teams (5–200) | Centralized billing, shared rules, org-wide privacy mode, RBAC, SAML/OIDC SSO, usage analytics | | Enterprise | Custom | Regulated, large orgs | Pooled usage, SCIM, audit logs, AI code tracking API, granular admin/model controls, SLA, dedicated AM | For most professional engineers Pro is enough; for full-time AI-driven development Pro+ or Ultra tend to pay back in saved credit anxiety. Teams becomes worth it as soon as you have shared rules, want one bill, or care about org-wide privacy enforcement. ### The 2025 transition to a credit pool — and the backlash Cursor's pricing structure today is the result of a contentious 2025 reset. [Vantage's pricing breakdown](https://www.vantage.sh/blog/cursor-pricing-explained) documents that "in June 2025, Cursor overhauled its pricing model, replacing fixed 'fast request' allotments with usage-based credit pools tied to actual API costs." The economic case was real — frontier-model usage was running away from any flat per-request structure — but, per the same write-up, "the rollout was rocky. Poor communication and unexpected charges led to a lot of community backlash," and Cursor "issued a public apology on July 4, 2025, offering refunds for unexpected charges incurred between mid-June and early July." The lasting effect is that the same $20 Pro that used to feel "all-you-can-eat" now visibly drains during heavy Opus 4.7 sessions. The current [Cursor pricing page](https://cursor.com/pricing) formalizes the answer to that complaint: Pro+ ($60) advertises "3x usage on all OpenAI, Claude, Gemini models" and Ultra ($200) advertises "20x usage" plus "priority access to new features." Teams ($40/user) adds shared chats, commands, rules, centralized billing, RBAC, SAML/OIDC SSO and usage analytics. Pricing-page text and per-credit costs do change — re-check before quoting numbers in a procurement deck. ## 4. Models in Cursor: built-in and custom Cursor's model picker exposes a curated list of frontier models that Cursor proxies on your behalf. As of May 2026 the most-used picks are: - **Claude Opus 4.7** — strongest model for architecture, deep refactors, and debugging gnarly logic. Higher credit cost; worth it on hard problems. - **GPT-5.5** — broad strength across coding tasks, fast, the safest "default." - **DeepSeek V4 Pro** — reasoning quality close to the frontier at roughly a tenth of the cost; great for high-volume work where you're paying yourself back per credit. See our [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) for details. - **Gemini 2.5 Pro** — long-context champion (1M+ tokens) for whole-repo passes. ### Custom OpenAI-compatible endpoints Cursor lets you bring any OpenAI-compatible API into the IDE: enter a name, base URL, and key, and the model appears in the picker marked with a person icon. Charges go directly to your provider account, bypassing Cursor's credit pool. Common reasons to do this: - You already have a self-hosted DeepSeek or Llama endpoint on your infra. - You want to point at OpenRouter or Together for a model Cursor hasn't curated yet. - You're running a privacy-sensitive setup and want every request to hit your VPC. Engineers we place often use this for cost-control pairings: a cheap custom endpoint for autocomplete-class tasks, frontier Cursor models reserved for agent runs. ## 5. Cursor Rules and `.cursorrules` Rules are the single biggest lever to make Cursor stop hallucinating in your codebase. The legacy `.cursorrules` file at repo root is now superseded by Project Rules: `.mdc` files inside `.cursor/rules/`, version-controlled per project, and scoped to specific globs. What good rules look like in 2026: - Reference canonical files instead of inlining patterns ("see `src/api/users/route.ts` for our handler shape"). - Explicit guardrails: never delete `.env` or `package.json` without confirmation; never commit without review; never assume a package exists without running `npm list <name>` first. - Verifiable goals: which lint, type-check, and test commands the agent must run before declaring done. - Tone and verbosity ("be concise; don't explain standard patterns"). - Stop conditions ("if you find a security issue, halt and report"). Teams that go from "30% of suggestions accepted" to "80%+" almost always made the jump by writing a focused rules file, not by upgrading models. Start small, add a rule when you watch the agent make the same mistake twice. ## 6. Codebase indexing, embeddings, and privacy mode When you open a repo, Cursor chunks files locally, computes a Merkle tree of file hashes, and syncs those hashes to its server. Embeddings are produced (OpenAI's embedding API or a custom embedder), stored in Turbopuffer (a remote vector DB), and used to retrieve semantically relevant chunks at query time. Re-indexing the same repo is fast because chunks are cached by hash. To preserve privacy without breaking path-based filtering, Cursor obfuscates file paths: each segment is split on `/` and `.` and encrypted with a client-held secret key. Even with the index "in the cloud," your folder structure isn't sitting plaintext in a vendor database. ### Privacy mode With Privacy Mode on, Cursor's backend doesn't retain code or data after fulfilling a request. No plaintext code is persisted server-side or in Turbopuffer; plaintext is fetched only at inference time, only for the specific files and lines a request needs, and discarded. Cursor maintains zero-data-retention agreements with all model providers it proxies. For regulated work, Privacy Mode is the floor; Enterprise adds enforced-org-wide and audit logging on top. If even Privacy Mode is too much trust to grant, you're outside Cursor's threat model and should be looking at fully-local alternatives — see our comparisons of [Cursor vs Void AI](https://codersera.com/blog/cursor-ai-vs-void-ai-an-in-depth-comparison-of-modern-ai-code-editors/) and [features, privacy, local models, and limitations in 2026](https://codersera.com/blog/void-ai-vs-cursor-features-privacy-local-models-limitations-2026/). ## 7. MCP support: connecting Cursor to your tools Model Context Protocol (MCP) is the standard for plugging external tools and data sources into AI clients. Cursor's MCP support has matured from "power-user toy" in 2024 to "first-class feature with one-click install" in 2026. How it shows up in practice: - **Three transport types:** stdio (simplest, local), SSE, and Streamable HTTP. Use stdio for CLI-shaped servers; HTTP for hosted ones. - **Two scopes:** global (Cursor Settings → Tools & MCP) or per-project via `.cursor/mcp.json` committed alongside your code. - **One-click installs** from a curated catalog (GitHub, Linear, Sentry, Postgres, Notion, Stripe, etc.) with OAuth flows handled inside Cursor. - **40-tool ceiling** per session — exceed it and Cursor stops exposing additional MCP tools to the model. Curate. The combination that consistently delivers the most value: a GitHub MCP server, a database MCP server (Postgres or your warehouse), and a docs-search server pointed at your internal Confluence/Notion. With those three, Cursor's agent stops needing to ask you for context it can fetch itself. ## MCP in 2026: "Add to Cursor", the 40-tool budget, servers worth wiring up The MCP onboarding flow tightened up in 2026. Per [Cursor's MCP docs](https://cursor.com/docs/mcp), you can now "click 'Add to Cursor' on a marketplace entry to install it and authenticate with OAuth" — one-click installs from the Cursor Marketplace and `cursor.directory` replace the old hand-edit-JSON dance. Configuration still lives in two scopes: `.cursor/mcp.json` in your repo for project-specific tools (commit it), and `~/.cursor/mcp.json` in your home directory for tools you want everywhere. The hard ceiling to plan around: third-party guides like [Truefoundry's MCP-in-Cursor guide](https://www.truefoundry.com/blog/mcp-servers-in-cursor-setup-configuration-and-security-guide) note that "Cursor has a ceiling of roughly 40 active tools across all your MCP servers combined." Past that limit, tool definitions blow your context budget and the agent silently loses access to later tools. Curate aggressively — disable servers you aren't actively using. The combinations engineers we place actually wire up: **GitHub** (PRs, issues, code search), **Postgres** or your warehouse (read-only by default), a docs-search server (Notion, Confluence), **Linear** or Jira for tickets, and **Playwright** when the agent needs to drive a browser. Five servers, well under 40 tools, and the agent stops asking you for context it can fetch itself. ## 8. Cursor in 2026: background agents, Bugbot, multitask The shape of Cursor changed materially in late 2025 and early 2026. Three releases worth knowing: - **Background agents (Cursor 1.0, mid-2025):** long-running cloud agents that can take a ticket, work for tens of minutes, and propose a PR. Now generally available on every paid tier. - **Bugbot (graduated to fixer, Feb 2026):** reviews PRs and, when it finds a real bug, spins up its own cloud agent, tests a fix, and proposes it directly on the PR. Resolution rate now around 80% — meaningfully ahead of competing review bots. - **Multitask / async subagents (Cursor 3.0):** the `/multitask` command farms a request out to parallel async subagents instead of queuing it. Combined with worktrees, you can run several isolated tasks across branches at once and pull whichever succeeds into the foreground. The practical shift: Cursor isn't just an editor anymore — it's an agent execution runtime that happens to have an editor attached. Pricing tiers track that shift; "credits" map to "agent-minutes" more closely every release. ## 9. Cursor vs the alternatives The honest comparison, after running each in production: | Tool | Form factor | Strongest at | Weakest at | Pricing (entry) | | --- | --- | --- | --- | --- | | Cursor | Forked VS Code IDE | Daily IDE workflow, Tab completion, MCP, team controls | Massive monorepo indexing, opinionated review surfaces | $20/mo Pro | | Claude Code | Terminal-based agent | SWE-bench-grade refactors, large-context reasoning, security audits | No IDE UX, no Tab autocomplete | $20/mo (Max tiers from $100) | | Windsurf | Forked VS Code IDE | Cascade context persistence, budget-friendly | Smaller MCP ecosystem; March 2026 price hike to $20 | $20/mo | | Cline | VS Code extension (open source) | Free, BYO model, transparent agent loop | You manage your own keys, ceilings, and prompts | Free | | Void AI | Fully open-source IDE | Local models, full data sovereignty | Smaller ecosystem; UX still maturing | Free | | Continue.dev | VS Code/JetBrains extension | Open-source, configurable, BYO model | Less polished agent flow than Cursor | Free | | Aider | CLI | Git-aware pair programming, scriptable workflows | No IDE; not great for exploratory work | Free (BYO model keys) | The pattern we see at engineering teams that we extend with vetted Codersera engineers: Cursor for daily IDE work, Claude Code in a terminal for big refactors and audits, Cline or Void as a local fallback when a client's data-residency policy forbids cloud inference. If you want fully-local setups, our walkthroughs cover [Void AI with Ollama on macOS](https://codersera.com/blog/run-void-ai-with-ollama-on-mac-best-cursor-alternative/), [on Ubuntu](https://codersera.com/blog/run-void-ai-with-ollama-on-ubuntu-best-cursor-alternative/), and [on Windows](https://codersera.com/blog/run-void-ai-with-ollama-on-windows-cursor-alternative/). ## 10. Real workflow examples ### Adding a feature to a Next.js app 1. Open Ask. "Where does the upload pipeline currently put files, and where do we generate signed URLs?" Skim the answer; correct any wrong assumptions. 2. Still in Ask: "Draft a plan to add server-side virus scanning before we generate the signed URL. Don't write code yet." 3. Switch to Agent with the plan in context. "Implement step 1 only. Stop after the unit test passes." 4. Review the diff line-by-line. Reject anything that touched a file outside the planned scope. 5. Repeat for steps 2 and 3. ### Refactoring a sprawling React component 1. Use Manual edit mode. Select the file. Prompt: "Extract the form-state logic into a custom hook, keeping all behavior identical. Don't change props or rendering." 2. Accept hunk by hunk; let Cursor regenerate hunks you don't like. 3. Run the typecheck and tests. If they pass, commit. ### Onboarding a new engineer to an unfamiliar codebase 1. Pin a Project Rule that points to the architecture doc and the canonical handler/component examples. 2. Have the engineer use Ask mode for the first week to build a mental model. Agent mode is off-limits until they can predict what it'll do. ## 11. Known issues and gotchas - **Long-session reasoning degradation.** Agent quality drops on very long single sessions — context starts to fragment, tool call counts balloon. Restart the session at natural seams (per task, per PR). - **Monorepo indexing.** Without a tuned `.cursorignore`, indexing a 1M-file monorepo can saturate disk IO for 5–15 minutes after opening. With one, it's 10–30 seconds. Always ignore `node_modules`, build artifacts, generated code, and any package directories irrelevant to your current scope. - **Cross-package leakage.** In monorepos, the agent will happily suggest importing backend code from a frontend package if it sees both indexed. Codify package boundaries in a Project Rule. - **Big sprawling diffs.** Multi-file refactors can produce PRs that are technically correct but practically unreviewable. Force the agent to work in smaller scopes; reject "while I was at it" changes. - **Hallucinated APIs.** Even Opus 4.7 invents methods that look right. Make "verify the package and method exist before calling" a rule. - **Credit burn.** Heavy Agent runs on Opus can chew through a $20 monthly pool in a few days. Either route routine tasks to a cheaper model (DeepSeek V4 Pro, GPT-5.5 mini) or upgrade to Pro+ / Ultra. - **Performance on huge files.** Files over ~5,000 lines slow Cursor down compared to vanilla VS Code. Split or refactor before letting the agent touch them. - **MCP tool ceiling.** 40 tools per session. Past that, the model stops seeing later tools. Curate ruthlessly. ## 12. Team and enterprise controls For organizations standardizing on Cursor: - **SSO:** SAML 2.0 (Okta, Azure AD, Google Workspace, generic). Local logins can be disabled. - **Provisioning:** SCIM for user lifecycle. - **Privacy:** Enforce Privacy Mode org-wide; Cursor maintains zero-data-retention agreements with proxied model providers. - **Compliance:** SOC 2 Type 2, GDPR, CCPA. Annual penetration testing, AES-256 at rest, TLS 1.2+ in transit. - **Admin levers:** repo allow/blocklists, model allow/blocklists, MCP server allow/blocklists, agent run defaults, AI code tracking API and audit logs (Enterprise). Most procurement objections to Cursor in 2026 are about model providers, not Cursor itself — handled with Enterprise's enforced privacy and on-prem options. ## FAQ ### Is Cursor worth $20/month? For a working engineer, yes — usually within the first week. The Tab autocomplete alone, paired with one well-scoped Agent task per day, pays back the seat. The break-even is much lower than seat cost. ### Do I need to migrate from VS Code? No painful migration. Cursor imports VS Code settings, keybindings, and extensions on first launch. You can keep both installed. ### Which model should I default to? Default to GPT-5.5 or Sonnet for everyday work; reach for Opus 4.7 on architecture and gnarly debugging; route bulk Tab-style tasks to DeepSeek V4 Pro to save credits. ### Can I use my own API keys? Yes, via custom OpenAI-compatible endpoints. Charges hit your provider account directly, bypassing Cursor's credit pool. ### Does Cursor train on my code? Not with Privacy Mode enabled, and not under enforced Enterprise privacy. Cursor also has zero-data-retention agreements with the model providers it proxies. ### How does codebase indexing actually work? Files are chunked locally, hashed into a Merkle tree, and the chunks are embedded and stored in a remote vector database (Turbopuffer). Paths are obfuscated via per-segment encryption. ### What's the difference between `.cursorrules` and Project Rules? `.cursorrules` at repo root is the legacy format. Project Rules (`.cursor/rules/*.mdc`) are the current standard — version-controlled, scoped to globs, and richer. ### Can Cursor work fully offline? No. Cursor requires cloud inference for chat and agent calls. If you need local-only inference, look at [Void AI](https://codersera.com/blog/cursor-ai-vs-void-ai-an-in-depth-comparison-of-modern-ai-code-editors/), Continue.dev with a local Ollama, or Cline pointed at a local model. ### How is Cursor different from Claude Code? Cursor is an IDE. Claude Code is a terminal agent. They overlap on agent capability but are complementary in practice — Cursor for in-flight editing, Claude Code for big async tasks. ### What's the right Cursor plan for a 10-engineer team? Teams ($40/user). You get shared rules, centralized billing, org-wide privacy enforcement, RBAC, SSO, and usage analytics. Below ~5 engineers, individual Pro often pencils out cheaper. ### Does Cursor support Jupyter notebooks? Yes, via the same VS Code Jupyter extension you'd use elsewhere. Agent edits work on cells, though large notebooks suffer from the same long-file performance hit as any 5k+ line file. ### What about Cursor's CLI? Cursor's CLI lets you start agent runs from a terminal, integrate with CI, and configure MCP from the shell. It's a complement to the IDE, not a replacement. ### Is Bugbot worth turning on? For most teams, yes. The 2026 fixer-grade Bugbot resolves close to 80% of the issues it raises and learns from PR feedback over time. Treat it as a junior reviewer that needs senior oversight, not a replacement for human review. ### How do I keep Cursor from going off the rails on a big refactor? Scope tightly, plan in Ask first, write a Project Rule that lists "don't touch X, Y, Z," restart the session per task, and review every diff. Treat the agent as a junior engineer with a long memory but no judgment. ## What this means for hiring The senior-level signal in 2026 isn't "writes clean code." Tab-completion and Agent mode mean almost any working engineer can produce passable code on a known stack. The differentiator is *orchestration*: who can scope a task narrowly enough to dispatch four to eight parallel agents with `/best-of-n`, review the resulting diffs critically, reject the "while I was at it" changes, and merge the right one. That's the same skill set that makes a strong tech lead reviewing five junior PRs — Cursor 3.0 just compresses the loop to minutes instead of days. When we vet engineers for client teams at Codersera, fluency with rules files, MCP curation, agent scoping, and worktree-based parallel runs is now part of the technical-fit evaluation, not a "nice to have." ## Next steps Cursor is a force multiplier when paired with engineers who know how to drive it — and a productivity sink when handed to engineers who treat it as autocomplete on steroids. The teams getting the most out of it have invested in rules, scoping discipline, and review culture, not just seat licenses. If you're scaling an engineering team and want senior, vetted, remote engineers who already work this way in production: [Hire a Codersera-vetted TypeScript or full-stack engineer](https://codersera.com/blog/hire/hire-typescript-developer) who works with Cursor in production. Risk-free trial, faster hiring, and engineers who can extend your team without extending your hiring risk. =============================================================== [13/20] Self-Hosting LLMs in 2026: The Complete Guide URL: https://codersera.com/blog/self-hosting-llms-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== Self-hosting large language models stopped being a hobbyist exercise sometime in late 2025. By May 2026, open-weight models (DeepSeek V4, Llama 4, Qwen 3.5, Gemma 4) match or beat the closed frontier on most non-reasoning workloads, inference engines like vLLM and SGLang have become genuinely production-grade, and Blackwell-class GPUs have collapsed the cost-per-token math against API providers. The question is no longer *can* you self-host — it is whether you should, what to run on, and how to keep latency tail-flat under real traffic. This guide is the engineering brief we hand teams at Codersera before they wire a self-hosted LLM into a product roadmap. *Last updated: May 1, 2026.* ## TL;DR - **Cost crossover vs frontier APIs lands around 2M–5M tokens/day** on reserved GPU capacity over a 12-month window. Below that, the API still wins. - **SGLang now leads vLLM by ~29% throughput on H100** (16,200 vs 12,500 tok/s on standard workloads) and up to 6x on prefix-heavy RAG pipelines thanks to RadixAttention. - **B200 delivers ~3x the throughput of H200 on Llama 2 70B Interactive** in MLPerf reporting, with HBM3e at 8 TB/s vs H200's 4.8 TB/s — and roughly 3x lower $/token in FP4 serving. - **FP8 is the production-default precision in 2026.** NVFP4 is rolling out on Blackwell but calibration tooling is still maturing; FP4 is not yet a safe default. - **DeepSeek V4 Flash needs ~158 GB of VRAM** in FP4+FP8 mixed precision — fits on a single H200, comfortable on 2x H200 for production-grade KV cache headroom. - **For dev workstations, the RTX 5090 (32 GB GDDR7, 1.79 TB/s) and Mac Studio M3 Ultra (up to 512 GB unified, 800 GB/s) cover almost every model below 70B** at usable interactive speed. ## 1. Why self-host in 2026, and when not to Self-hosting earns its keep on three axes: **unit economics at volume**, **data residency**, and **latency control**. It loses on engineering overhead and on the speed at which you can iterate on a model upgrade. The honest break-even framing: if your application processes under ~50M tokens per month, almost every cost analysis we've reviewed concludes that hosted APIs are cheaper once you account for engineering time, on-call burden, and electricity. The crossover gets serious around 11 billion tokens per month — at that scale, self-hosted infrastructure with appropriate utilization comfortably undercuts GPT-4.1 / Claude 4 tier API pricing, sometimes by 4–10x. Three non-cost reasons we still see teams self-host below the crossover: - **Regulated data (healthcare, finance, defense).** The rules don't care about your token count. - **Latency-sensitive inline UX** (autocomplete, voice, real-time agents) where the 60–250 ms of round-trip API latency is the product. - **Custom adapters or fine-tunes** that simply aren't available on hosted endpoints. If none of those apply and you're under 2M tokens/day, keep using the API and revisit in six months. If one of them applies, the rest of this guide is for you. ## 2. Hardware tiers: what to actually buy or rent The hardware decision splits cleanly into **workstation** (one developer, one model, interactive use), **single-node production** (one model serving real traffic), and **multi-node fleet** (multiple models, autoscaling, redundancy). Don't conflate them — the right answer for each is wildly different. ### Workstation tier The RTX 5090 has reset the local-inference bar. With 32 GB of GDDR7 and 1.79 TB/s of bandwidth (a 78% jump over the 4090), it hits roughly 234 tok/s on a 30B MoE at short context, over 10,000 tok/s on prefill for Qwen3 8B, and well past 17,000 generation tok/s on Qwen3-8B Q4_K_M with speculative decoding tricks. For most 7B–32B work, one 5090 is enough. Two 5090s comfortably serve a 70B model in Q4 quantization and beat an A100 on $/token. The Mac Studio M3 Ultra is the surprise winner for memory-heavy work. Up to 512 GB of unified memory at 800 GB/s means a fully unquantized 70B fits trivially, and you can hold MoE models well past 100B in aggressive quantization. MLX (Apple's native framework) consistently runs 26–30% faster than Ollama on the same hardware. We cover the model-by-model tradeoffs in our deep-dive on the [best small LLMs for local hardware](https://codersera.com/blog/best-small-llms-to-run-locally-a-comprehensive-guide/) and the [DeepSeek V4 Flash local setup walkthrough](https://codersera.com/blog/run-deepseek-v4-flash-locally-full-2026-setup-guide/), both of which lean heavily on MLX for Mac users. ### Single-node production tier This is where H100 / H200 / B200 live. The H200 (141 GB HBM3e, 4.8 TB/s) gives 1.83x–2.14x the long-context throughput of an H100 across DeepSeek, Llama, and Qwen flagships. Two H200 SXM in one pod (282 GB pooled) is the current sweet spot for V4 Flash with 256K-context KV cache headroom. The B200 (192 GB HBM3e, 8 TB/s) is the new performance king — MLPerf reporting shows roughly 3x H200 throughput on Llama 2 70B Interactive and sub-3 ms latency on 8x B200 systems. SemiAnalysis InferenceX numbers put B200 at roughly 3x lower cost-per-token than H200 in FP4 serving, despite higher hourly rental. ### Hardware comparison | Tier | GPU / system | VRAM | Bandwidth | Realistic workload ceiling | Indicative $/hr (rented) | | --- | --- | --- | --- | --- | --- | | Hobbyist | RTX 4090 | 24 GB GDDR6X | 1.0 TB/s | 13B FP16, 32B Q4 | ~$0.35 | | Workstation | RTX 5090 | 32 GB GDDR7 | 1.79 TB/s | 32B FP16, 70B Q4 | ~$0.69 | | Workstation (memory-rich) | Mac Studio M3 Ultra 512 GB | 512 GB unified | 800 GB/s | 70B FP16, 120B+ Q4 MoE | n/a (capex ~$10k) | | Single-node prod | H100 SXM 80 GB | 80 GB HBM3 | 3.35 TB/s | 70B FP8, 100B+ MoE Q4 | ~$2.5–3.0 | | Single-node prod | H200 SXM 141 GB | 141 GB HBM3e | 4.8 TB/s | V4 Flash FP4+FP8 (2x), Llama 4 405B Q4 (2x) | ~$3.5–4.5 | | Frontier prod | B200 | 192 GB HBM3e | 8.0 TB/s | V4 full FP4, frontier MoE | ~$6–8 | Two warnings on the rental numbers: spot pricing varies 30–50% week to week, and several providers (Lambda, RunPod, Modal, Together, Anyscale) trade leadership on different SKUs. Always price two providers before committing. ## 3. Inference stacks: pick the engine for the workload, not the brand The inference engine matters more than the GPU once you're past the workstation tier. There are now seven serious options. They are not interchangeable. ### vLLM vLLM is the reference production engine. PagedAttention KV cache, continuous batching, OpenAI-compatible API, the broadest model support, and the largest community. v0.6.0 delivered a 2.7x throughput improvement and 5x latency reduction over earlier releases. On H100 with FlashInfer enabled, it peaks around 12,500 tok/s on standard workloads. The new `production-stack` sub-project ships a Helm chart, Prometheus metrics, KV-cache reuse via LMCache, and Grafana dashboards out of the box. If you don't have a strong reason to pick something else, pick vLLM. ### SGLang SGLang's RadixAttention beats vLLM by ~29% on standard H100 throughput (16,200 vs 12,500 tok/s) and up to 6x on prefix-heavy workloads — multi-turn chat, RAG over shared documents, few-shot prompting — where it sustains 75–95% prefix-cache hit rates. On DeepSeek V3-class MoE models, SGLang is ~3.1x faster than vLLM. The downside: smaller community, slightly less smooth Day 0 support for new model architectures, and a steeper learning curve for the structured-generation API. ### llama.cpp The C++ engine that almost every consumer-facing tool wraps. GGUF format, ggml kernels, runs on essentially anything (CUDA, Metal, Vulkan, ROCm, CPU). For single-stream inference at the workstation tier, llama.cpp is consistently the fastest option, especially with speculative decoding and the right Q4_K_M / Q5_K_M / Q6_K quantization. It does not do continuous batching well — you don't want it as your serving layer in production. ### Ollama Ollama is llama.cpp wrapped in a daemon, a CLI, an HTTP API, and a sane model registry. As of April 2026, it is the lowest-regret entry point for local inference: `ollama pull qwen2.5:32b-instruct-q4_K_M` and you have an OpenAI-compatible endpoint on localhost in 30 seconds. We use it as the runtime in our [local agent setup guide](https://codersera.com/blog/openclaw-ollama-setup-guide-run-local-ai-agents-2026/) and our [personal AI assistant walkthrough](https://codersera.com/blog/openclaw-with-ollama-run-personal-ai-assistant-local-models/). Don't use Ollama for serving real concurrent traffic — it's optimized for one-user-at-a-time, not for batched throughput. ### LM Studio The desktop GUI. Best in class on Apple Silicon thanks to first-class MLX support; the only one of these tools where double-clicking a model file works. We compare it against the alternatives in detail in our writeup on [LM Studio vs Ollama vs OpenClaw](https://codersera.com/blog/openclaw-vs-lm-studio-vs-ollama-best-local-ai-workflow-developers/). Production-irrelevant — no Docker support, desktop-only. ### MLX Apple's framework. If you're on Mac, this is what your stack should be calling under the hood. 26–30% faster than Ollama on M3 Ultra for the same model and quantization. Native FP16, BF16, INT4, and now FP8 support on M-series. ### TGI (Text Generation Inference) HuggingFace's serving stack. As of mid-2025 it moved into maintenance mode — minor bug fixes only. It still works fine for HuggingFace Inference Endpoints customers, but new deployments should default to vLLM or SGLang. TGI now multi-backends through TRT-LLM and vLLM under the hood anyway. ### Inference engine comparison | Engine | Best for | Throughput (H100, std) | Continuous batching | Prefix caching | Operational maturity | | --- | --- | --- | --- | --- | --- | | vLLM | General production serving | ~12,500 tok/s | Yes | Yes (LMCache) | High | | SGLang | RAG, multi-turn, structured output | ~16,200 tok/s | Yes | Yes (RadixAttention) | Medium-High | | TensorRT-LLM | NVIDIA-only, max FP8/FP4 perf | ~16,200 tok/s | Yes | Partial | High (NVIDIA-supported) | | llama.cpp | Single-user, broad hardware | n/a (single-stream tool) | Limited | Manual | High (community) | | Ollama | Dev environments, agents | Inherits llama.cpp | Limited | No | Medium | | LM Studio | Mac desktop | Inherits llama.cpp / MLX | No | No | Low (desktop) | | MLX | Apple Silicon production lite | n/a (Apple-only) | Limited | Yes | Medium | | TGI | HF Inference Endpoints | Comparable to vLLM | Yes | Yes | Maintenance mode | ## 4. Models: which open weights to actually run The 2026 open-weight tier is a four-horse race: **DeepSeek V4 / V4 Flash** (March–April 2026), **Qwen 3.5** (February 2026, Apache 2.0), **Gemma 4** (April 2026, Apache 2.0), and **Llama 4** (custom community license, MoE). All four families ship MoE flagships; the dense models are now the smaller variants. For a typical product team, the decision tree is simpler than the leaderboards make it look: - **Reasoning, code, complex agents:** DeepSeek V4 or V4 Flash. V4 Flash gives you ~80% of full V4 quality at ~25% of the VRAM. Our [DeepSeek V4 Flash deep dive](https://codersera.com/blog/deepseek-v4-flash-deep-dive/) and the broader [V4 pillar](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) walk through the architecture and the FP4+FP8 mixed-precision deployment story. - **General chat, RAG, balanced workloads:** Qwen 3.5 27B (dense) or 397B-A17B (MoE). Apache 2.0, no licensing pain. Edges Gemma 4 31B on MMLU Pro (86.1% vs 85.2%) and GPQA Diamond (85.5% vs 84.3%). - **Math, competitive code, structured reasoning:** Gemma 4 31B leads the pack — AIME 2026 at 89.2%, Codeforces ELO of 2150. The 26B-A4B MoE activates only 3.8B parameters per forward pass and lands 6th on Arena AI text leaderboard at 1441 ELO. Best efficiency-per-parameter in the open ecosystem. - **Long-context document workflows:** Llama 4 Scout. 109B params, 17B active, 10M-token theoretical context (1–2M practical). License caveat: not usable in apps with over 700M MAU. - **Edge / sub-16 GB hardware:** Phi-4 (14B) for math and structured reasoning — 80.4% on MATH, beats many 30–70B models. Phi-4-mini (3.8B) at Q4_K_M runs at 15–20 tok/s on an M1 Air. Mistral Small 3 (24B dense) beats Llama 3.3 70B at a third of the params and runs on a single 4090 quantized. We have a survey of these in our [small-LLM guide](https://codersera.com/blog/best-small-llms-to-run-locally-a-comprehensive-guide/). - **Massive model on tiny hardware:** oLLM-style CPU offload tricks — covered in our writeup on [running an 80 GB model on 8 GB of VRAM](https://codersera.com/blog/how-to-run-80gb-ai-model-locally-on-8gb-vram-ollm-complete-guide/). Slow, but real. ## 5. Quantization: what's safe to ship in production Four quantization regimes matter in 2026. **FP16 / BF16** — the reference. Use when VRAM allows. No accuracy concerns, slowest throughput. **FP8** — the production default on Hopper and Blackwell. Typical accuracy degradation is 0.5–2% on standard benchmarks; head-to-head, Llama-3.1-70B at FP8 scores 69.64% on MMLU-Pro vs 70.24% BF16 (a 0.6 point gap), and matches BF16 exactly on HumanEval at 39.02%. Use FP8 by default for serving on H100/H200/B200. **INT4 (AWQ, GPTQ, GGUF Q4_K_M)** — the consumer-hardware default. AWQ retains roughly 95% of base quality, GPTQ around 90%. The Marlin kernel made INT4 actually fast on modern GPUs: Marlin-AWQ hits 741 tok/s vs vanilla AWQ's 68 tok/s (10.9x speedup); Marlin-GPTQ hits 712 tok/s vs 276 tok/s (2.6x). Sources disagree on which method wins on which task — AWQ tends to win on code, GPTQ tends to win on instruction-following (IFEval). Test on your eval set before committing. **FP4 (NVFP4, MXFP4)** — the Blackwell unlock. NVIDIA TensorRT Model Optimizer with NVFP4 gives near-FP8 quality at half the memory and double the FLOPS on B200. As of May 2026, FP4 is not the production default — calibration tooling is still maturing, and accuracy varies by model and task. Treat it as a high-value bet for cost-sensitive Blackwell deployments where you can validate end-to-end on real evals. ### Quantization quick-pick | Format | Bits | Accuracy retention | Where it shines | Where to avoid | | --- | --- | --- | --- | --- | | BF16/FP16 | 16 | 100% | Reference, training | Cost-sensitive serving | | FP8 | 8 | 98–99.5% | Hopper/Blackwell production | Pre-Hopper hardware | | AWQ-INT4 (Marlin) | 4 | ~95% | Consumer GPUs, code workloads | Strict instruction-following | | GPTQ-INT4 (Marlin) | 4 | ~90% | Cheap memory savings, IFEval | Code-heavy production | | GGUF Q4_K_M | ~4.5 | ~94% | llama.cpp / Ollama / LM Studio | Multi-tenant serving | | NVFP4 / MXFP4 | 4 | ~96–98% (model-dependent) | B200 cost optimization | Anything you can't re-evaluate | ## 6. Serving patterns: dev vs prod ### Single-GPU developer pattern One model, one GPU, OpenAI-compatible endpoint on localhost. The whole stack is `vllm serve Qwen/Qwen3.5-27B-Instruct --quantization fp8` or the equivalent `ollama run` command. Use it for prototyping, evals, and integration tests. Don't ship it. ### Multi-GPU production pattern The minimum production stack as of May 2026: - 2x H200 (or 1x B200) running vLLM or SGLang in tensor-parallel mode - An L4 or L40S in front for the embedding model (cheap, fast, separate KV pressure) - Kubernetes with the vLLM `production-stack` Helm chart, or an equivalent on Modal / RunPod / Anyscale - LMCache for cross-request KV reuse — typically 30–60% effective throughput uplift on RAG-shaped traffic - A token-aware autoscaler keying off `vllm:num_requests_waiting` and KV-cache utilization, not CPU% - Speculative decoding for latency-sensitive paths: a 1B–3B draft model in front of the production model, gives 2–3x speedup on greedy decoding ### Hybrid pattern (the underrated default) Run a self-hosted instance for high-volume, predictable workloads (embeddings, classification, summary, retrieval rewrites), and route long-tail or peak-load requests to a hosted API. Most teams over-build the self-hosted side and underestimate how much spiky traffic the API absorbs cheaply. Don't be one of them. ## 7. Monitoring and observability LLM serving has its own metrics that traditional APM doesn't capture. The non-negotiables: - **End-to-end latency** (request received → final token) — your customer-facing SLO. - **Time to first token (TTFT)** — dominated by prefill. Spikes mean queue depth or KV pressure. - **Inter-token latency (ITL)** — dominated by decode. Spikes mean batch contention or memory bandwidth saturation. - **KV cache utilization** — vLLM exposes this directly. When it crosses ~85%, latency tail blows up. This is your north-star gauge. - **Queue depth (`num_requests_waiting`)** — your autoscaler signal. - **Tokens generated / second** — capacity-planning metric, not an SLO. vLLM exposes all of this on `/metrics` in Prometheus format. Pair with Grafana dashboards (the production-stack ships defaults), wire in OpenTelemetry traces for the request lifecycle, and you'll be able to debug a tail-latency regression in minutes rather than hours. ## 8. Cost crossover, in detail The math, as cleanly as we can state it for May 2026: - **Under 50M tokens/month:** APIs win. Don't self-host. - **50M–500M tokens/month:** A reserved 2x H200 node serving Qwen 3.5 or DeepSeek V4 Flash at ~70% utilization is roughly cost-neutral with frontier APIs. The win comes from latency control or data residency, not the bill. - **500M–11B tokens/month:** Self-hosting wins on cost. Plan for 30–50% savings vs frontier APIs after engineering overhead. - **11B+ tokens/month:** Self-hosting wins decisively. 4–10x savings are realistic, especially on B200 with FP4 / NVFP4. One trap to avoid: comparing self-hosting against the *frontier* API price (GPT-4.1, Claude 4) when your workload would run fine on a cheaper hosted tier (Haiku, Gemini Flash, DeepSeek API). The API providers also have a Flash tier; the crossover against *that* is much further out — closer to 200M+ tokens/day. ## 9. Known issues and sharp edges - **KV cache fragmentation under heterogeneous request lengths.** vLLM 0.6+ helps, but you can still see 20–30% throughput loss when mixing 200-token and 32K-token requests in the same batch. Solution: route by length, or pin a separate replica for long-context. - **Speculative decoding interacts badly with structured output.** If you're using grammar-constrained generation or JSON-mode, draft models often fail validation, killing the speedup. Disable speculative for those paths. - **FP8 KV cache is not free.** Some models lose noticeable quality with FP8 KV (vs FP8 weights only). Check on your evals before flipping it on. - **NCCL across vendor / driver mismatches.** Mixing GPU generations in one tensor-parallel group is a debugging nightmare. Don't. - **Tokenizer drift.** Quantized models occasionally ship with subtly different tokenizers. Compare token counts on a fixed prompt before promoting. - **Cold start on consumer hardware.** Loading a 70B Q4 model from disk to GPU can take 60–120 seconds. Pre-warm before traffic. - **Mac Studio thermal throttling under sustained load.** Real, but only matters past 30+ minute heavy generation sessions. Unlikely to bite interactive use. - **Ollama defaults to a tiny context window** (often 2k or 4k) regardless of what the model supports. Override with `OLLAMA_CONTEXT_LENGTH` or modelfile `num_ctx`. ## 10. FAQ ### How much VRAM do I actually need for a 70B model? FP16: ~140 GB plus KV cache. FP8: ~70 GB. Q4 (AWQ/GPTQ/GGUF): ~35–40 GB plus KV. With Q4 you can run a 70B comfortably on a single H100 80 GB, or on 2x RTX 5090s with tensor parallelism. ### Is DeepSeek V4 Flash actually self-hostable? Yes, on 1x H200 (141 GB) with FP4+FP8 mixed precision at ~158 GB on-disk weights — tight but viable. 2x H200 SXM (282 GB pooled) is the production-grade configuration with comfortable KV headroom for 256K context. ### Should I use vLLM or SGLang? Default to vLLM. Switch to SGLang if your workload is RAG, multi-turn chat, or anything with heavy prefix reuse — that's where the 6x throughput edge shows up. ### Is Ollama production-ready? For a single team's internal tools, yes. For multi-tenant customer-facing serving, no — it's optimized for one user at a time. Use vLLM or SGLang for that. ### What's the cheapest way to try a 405B-class model? Rent a single 8x H100 or 4x H200 node from RunPod or Lambda for an hour. Spin up vLLM with the model in FP8, hit it from your laptop, tear it down. Total cost: under $30. ### Can I really run a 70B on a Mac? Yes — comfortably on M2 Ultra / M3 Ultra with 96 GB+ unified memory. MLX or LM Studio give you 8–15 tok/s on a Q4 70B at usable interactive latency. ### How does FP8 compare to FP4 in practice? FP8 is the boring, safe production choice in 2026. FP4 (NVFP4) on Blackwell is the cost lever for high-volume serving, but plan to validate end-to-end on your eval set — calibration tooling isn't bulletproof yet. ### Do I need Kubernetes? Below ~3 nodes, no — Docker Compose or systemd plus a load balancer is fine. Above that, the autoscaling and rollout story gets painful without it. The vLLM production-stack Helm chart is the path of least resistance. ### What about AMD MI300X / MI325X? vLLM and SGLang both support ROCm, and MI300X has 192 GB of HBM3 — more than an H100. Real-world throughput trails NVIDIA on most kernels by 10–30%, but $/token is competitive on the secondary cloud market. Worth pricing if NVIDIA capacity is unavailable. ### How do I handle model upgrades without downtime? Blue-green at the deployment level: bring up the new model on a parallel replica set, shift traffic via the load balancer, drain the old set. The vLLM production-stack supports this natively. ### What's the simplest stack for a startup hitting product-market fit? Hosted API for the user-facing path, plus one self-hosted Ollama or vLLM instance on a rented 5090 or H100 for embeddings and classification. Move more inline as your token volume crosses 50M/month. ### Is fine-tuning worth doing in 2026? For most teams, no — well-prompted Qwen 3.5, Gemma 4, or DeepSeek V4 Flash will match a fine-tuned 7B on most tasks. Fine-tune when you have a domain-specific eval that pretrained models reliably fail and you have enough labeled data (10k+ high-quality examples) to move it. ### How do I prevent KV cache OOM under load? Set `--max-num-batched-tokens` and `--gpu-memory-utilization` conservatively (start at 0.85), monitor `vllm:gpu_cache_usage_perc`, and reject or queue requests when it crosses 0.85. Don't try to use 100% of VRAM — fragmentation will get you. ### What's the right way to benchmark my own deployment? Replay real traffic, not synthetic. Capture a week of production prompts (with PII scrubbed), replay against the candidate stack at 1x, 2x, and 5x rate, and measure P50/P95/P99 TTFT, ITL, and end-to-end latency. Synthetic benchmarks lie about prefix cache hit rates. ## Next steps If you're under 2M tokens/day, stay on the API and use this guide as a six-month checkpoint. If you're crossing the threshold, the highest-leverage next moves are: (1) pick one model family and one inference engine and stick with them long enough to learn their failure modes, (2) instrument `/metrics` end-to-end before you scale, and (3) run a real-traffic replay benchmark on your candidate stack before you commit to multi-node infrastructure. Most teams that successfully ship self-hosted LLMs do it with two engineers: one who has shipped production GPU infrastructure before, and one who has shipped applied ML. If you don't have both already, that's the gap to close first. [Hire a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer) who has shipped self-hosted LLMs. =============================================================== [14/20] Apple Silicon LLMs: Complete Guide to Running Models on Mac (2026) URL: https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/ Last updated: 2026-05-31 =============================================================== **Quick answer.** In May 2026, the Mac is a credible local-LLM box. Apple Silicon's unified memory means a 64 GB MacBook runs models that won't fit on a 24 GB RTX 4090, and MLX — Apple's native ML framework — has become the fastest way to run them, beating llama.cpp by 30–40% on M5 hardware. For most users the right stack is Ollama 0.19+ (which now uses MLX under the hood on Apple Silicon) for everyday chat and agent work, MLX-LM directly when you want maximum performance or to fine-tune, and LM Studio when you want a GUI. A 32 GB M-class Mac runs 30B mixture-of-experts models at ~100 tokens/sec; a 64 GB Mac runs 70B at usable speeds; multi-Mac clusters over Thunderbolt 5 now run frontier 120B+ models for sovereign teams. This guide covers what to buy, what to install, which models to download, and how to think about quantization, fine-tuning, and clustering on Apple Silicon. ## Why run LLMs locally on a Mac in 2026? Three things changed in 2024–2026 that turned the Mac from a curiosity into the default local-LLM machine for individual developers and small teams. **Unified memory matured into an unfair advantage.** On a discrete-GPU PC, the model has to fit in the GPU's VRAM — typically 8 to 24 GB on consumer hardware, and the model weights have to be copied across PCIe to get there. On Apple Silicon, the CPU and GPU share the same pool of memory at full bandwidth. A 64 GB MacBook Pro can load a 70-billion-parameter model at 4-bit quantization and start serving tokens in seconds, no copying. The 24 GB VRAM ceiling that limits a $1,600 RTX 4090 simply doesn't apply. **MLX won.** Apple's open-source MLX framework, released late 2023, hit production maturity in 2025 and pulled decisively ahead of llama.cpp's Metal backend in 2026. On the M5 generation, MLX is 30–60% faster on most workloads and 3–4× faster on prompt processing thanks to the Neural Accelerators embedded in every GPU core. The Hugging Face `mlx-community` org now hosts ~4,800 pre-converted models. Ollama, the most popular local-LLM CLI, switched its Apple Silicon backend to MLX in version 0.19 in March 2026 — a quiet but enormous performance win for anyone already using it. **The model landscape moved.** 2026 produced a wave of mixture-of-experts (MoE) open-weight models — Qwen 3.5 / 3.6 / 3.7, DeepSeek V4 Flash, Kimi K2.6, Mixtral families — that ship a small "active" parameter budget (3B–17B activated per token) on top of a large total parameter pool. They run fast on Mac because only the active experts have to be loaded into the working set, and Apple Silicon's bandwidth handles MoE routing well. The result: a 32 GB Mac running Qwen3-Coder-30B-A3B is the practical equivalent of a much bigger GPU box for most chat and code-completion workloads. None of this means a Mac replaces a multi-GPU cloud setup for serving paying customers — it doesn't. But for privacy-sensitive work, offline use, agent prototyping, learning, and the entire "I want to use a strong model without an API bill" pattern, the Mac in 2026 is the answer. ## What is the Apple Silicon advantage, exactly? Three hardware properties drive everything: **1. Unified memory architecture (UMA).** On the M-series chip, the CPU, GPU, and Neural Engine share a single pool of RAM with no copies between them. When you load a 40 GB model on a 64 GB Mac, every component can read those weights at full memory bandwidth — typically 400–500 GB/s on a Max-class chip, 800+ GB/s on Ultra. By contrast, an NVIDIA GPU has to copy weights from system RAM into VRAM across PCIe (~64 GB/s on PCIe 5.0 x16) before it can use them, and if the model doesn't fit in VRAM at all, it can't run. **2. Memory bandwidth that's actually high for the price.** Token generation in transformers is memory-bandwidth-bound; almost no compute matters compared to "how fast can I read the weights for the next token." Bandwidth ranks roughly: M-class Pro ≈ 273 GB/s, Max ≈ 500 GB/s, M5 Max ≈ 614 GB/s, M2/M3 Ultra ≈ 800 GB/s. An RTX 4090 manages 1,008 GB/s but only on the 24 GB that fits inside it. **3. Neural Accelerators on M5.** Apple's M5 chip embeds dedicated AI accelerators inside *every GPU core* (40 in M5 Max), and MLX uses them automatically. Apple's January 2026 paper showed Qwen3-14B at 4-bit running 4.06× faster on time-to-first-token and 1.19× faster on token generation versus M4 — and that's the accelerator contribution alone, before the bandwidth gain. For interactive chat and long-context agent workloads, M5 is a genuine generational step, not just a refresh. The catch: bandwidth governs decode speed, accelerators govern prefill speed. M5's 3.5× advertised "AI gain" is mostly about prefill (the model reading your prompt). Token-by-token output speed scales with bandwidth, which only improved ~15% from M4 to M5. Don't expect a 3.5× speedup on every workload — expect a dramatic speedup on time-to-first-token for long prompts, especially in agent loops. ## What is MLX and why did it win? **MLX** ([github.com/ml-explore/mlx](https://github.com/ml-explore/mlx)) is an array-computation framework for Apple Silicon, built and open-sourced by Apple in late 2023. Think NumPy + autograd + a Metal backend, with Python and Swift APIs. Around it, `mlx-lm` ([github.com/ml-explore/mlx-lm](https://github.com/ml-explore/mlx-lm)) provides LLM-specific tooling: model loading, generation, quantization, LoRA fine-tuning, and a tokenizer wrapper. MLX's design choices that mattered: a lazy computation graph (no eager-mode overhead), zero-copy weights through unified memory (no CPU↔GPU shuttling), and a function-transform model (`grad`, `vmap`, `compile`) that lets the framework fuse operations into one Metal kernel. The result is consistently faster than llama.cpp's Metal backend on the same model + quant, with the gap widening on M5 because MLX explicitly uses the Neural Accelerators that llama.cpp doesn't. Where it stood as of May 2026: `mlx` v0.31.x, `mlx-lm` v0.31.x, roughly a release every 3–4 weeks. MoE is fully supported (the kernel work landed in late 2025). Quantization tooling covers 3-bit through 8-bit plus mixed-precision modes (the `mxfp8` and `nvfp4` formats from the broader 4-bit-floating-point ecosystem). Distributed inference and training across multiple Macs via `mx.distributed` is production-shaped. The `mlx-community` Hugging Face org has ~4,810 converted models from ~4,830 community members — pick a model, append `-4bit` or `-8bit`, and it usually exists. MLX also ships a CUDA backend (~9% of the codebase is CUDA-targeted), so the framework isn't strictly Apple-only — it can run on Linux + NVIDIA. Practically, you wouldn't use MLX on NVIDIA over vLLM or TensorRT-LLM, but it broke the "MLX locks you in" framing. ## Pick your stack: MLX-LM, Ollama, LM Studio, llama.cpp, or vllm-mlx? The Mac local-LLM stack used to be a confusing five-way choice. In 2026 it's much cleaner, because most of these now share MLX under the hood: | Tool | What it is | Right when | | --- | --- | --- | | **Ollama 0.19+** | CLI + REST server, MLX backend on Apple Silicon since March 2026 | Everyday chat, agent workflows, you want a one-line install and an OpenAI-compatible API. The default for most users. | | **MLX-LM** | Apple's official Python CLI: `mlx_lm.generate`, `mlx_lm.server`, `mlx_lm.lora` | Maximum speed, fine-tuning, scripting in Python, custom inference loops. The right answer when Ollama is too opinionated. | | **LM Studio** | Desktop GUI that wraps both GGUF (llama.cpp) and MLX backends, plus a model marketplace | You want a GUI, want to browse and download models visually, want MCP server support, or are setting up a non-technical user. | | **llama.cpp / GGUF** | The cross-platform C++ reference, Metal backend on Mac | A model is brand-new and only has GGUF conversions yet, or you want truly portable inference code that runs on Mac/Linux/Windows from one binary. | | **vllm-mlx** | vLLM's API and PagedAttention, MLX as the kernel layer | You're serving multiple concurrent users / agent fleets and need batched throughput. Worse single-user speed, much better total throughput. | Independent benchmarks on an M4 Pro 64 GB running DeepSeek V3 at Q4 in May 2026 showed: Ollama 0.19+ ≈ 58 tokens/sec single-user with ~45 ms time-to-first-token (the best for interactive chat); vllm-mlx ≈ 42 tokens/sec single-user but ~1,150 tokens/sec aggregate at 32 concurrent users (the best for agent fleets); llama.cpp Metal ≈ 52 tokens/sec, the slowest of the MLX-backed group but still the broadest model support. The "Ollama is slow on Mac" critique that was true through 2025 is now mostly false — upgrade to 0.19 or later. Jan and GPT4All are also valid choices for the "fully offline desktop app, no telemetry" niche; both lag MLX-native tools on raw speed but are excellent for privacy-strict users. ## What can I actually run on my Mac? The honest buying-guide table — what runs comfortably at 4-bit (Q4) quantization, with expected tokens/sec on an M4 Max class chip via MLX. M5 numbers run ~15% faster on token generation and dramatically faster on time-to-first-token: | Unified RAM | Realistic ceiling | Representative model | Speed (M4 Max, Q4) | | --- | --- | --- | --- | | 8 GB | 3–4 B dense, small MoE | Gemma 4 E2B, Phi-3.5-mini | 60–90 tok/s | | 16 GB | 7–8 B dense | Llama 3.2 7B, Mistral 7B | 50–70 tok/s | | 24 GB | 13–14 B dense | Qwen 3.5 14B | 30–45 tok/s | | 32 GB | **30B MoE — the 2026 sweet spot** | Qwen3-Coder-30B-A3B | **~130 tok/s** | | 48 GB | 30–40 B dense | Command-R 35B | 18–25 tok/s | | 64 GB | 70 B Q4 comfortably | Llama 3.1 70B, Qwen3.5-122B-A10B | 12–15 tok/s on 70B | | 96–128 GB | 70 B bf16, 8x22B MoE | Mixtral 8x22B, DeepSeek V4 Flash (aggressive quant) | 8–15 tok/s on 70B FP16 | | 192 GB Ultra | 405 B at Q2, DeepSeek V4 at Q3 | Llama 3.1 405B Q2_K | 3–6 tok/s | | 512 GB Studio Ultra | DeepSeek V4 at usable quant | DeepSeek V4 Q4 | 15–25 tok/s | | Multi-Mac JACCL cluster | 120B+ at higher precision | DeepSeek V4 bf16, Llama 4 400B | 10–20 tok/s, scales with cluster | **The sweet spot for most developers in May 2026 is a 64 GB M4 Max or 64 GB M5 Max.** It runs the 30B-A3B MoE class at over 100 tokens/sec (faster than most cloud APIs for interactive work), runs 70B dense at acceptable speed, and leaves headroom for the context window — which matters more than people realise, because a 128 K-token context can easily eat 8–12 GB on top of the model weights. If you have an 8 GB Mac, you can still run something useful: Gemma 4 E2B, Phi-3.5-mini, or a tiny Qwen 3.5 variant. These are not GPT-5.5-replacements, but they handle classification, summarisation, light coding assistance, and structured-output tasks well, and they run completely offline. ## Which models should I download right now? The May 2026 picks, organised by what you're trying to do. All have native MLX conversions in `mlx-community`: **Best general-purpose chat model on 32 GB:** Qwen 3.6 35B-A3B MoE. Smart, fast, multilingual, instruction-following is on par with hosted Claude Sonnet for everyday tasks. **Best coding assistant on a Mac:** Qwen3-Coder-30B-A3B at 4-bit. Roughly 130 tok/s on a 64 GB M4 Pro, ~230 tok/s on M5 Max. The state of the art for offline code completion in 2026. See our [AI coding agents guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) for context on how it compares to Claude Code and Cursor. **Best reasoning model on 64 GB:** DeepSeek V4 Flash at Q4 (284B total / 13B active MoE). Released late April 2026 with MIT license. Punches above its weight class on math and coding benchmarks. The full DeepSeek V4 Pro (1.6T total) only fits on a 512 GB Mac Studio Ultra or a small cluster — see our [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). **Best 70B-class workhorse on 96 GB:** Llama 4 Maverick at Q4. The most-deployed local-LLM size for serious work. Stable, well-tuned, fine-tunable. See our [Llama 4 guide](https://codersera.com/blog/llama-4-complete-guide-2026/). **Best long-context model:** Kimi K2.6 at aggressive quant on Mac Studio Ultra, or Llama 4 Scout for its 10M-token context. Kimi K2.6 leads agentic benchmarks. See our [Kimi K2.6 guide](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/). **Best tiny model for 8–16 GB Macs:** Gemma 4 E2B / E4B. Google's small-model family, multimodal, 256K context. See our [Gemma 4 guide](https://codersera.com/blog/gemma-4-complete-guide-2026/). **Best vision-capable model:** Qwen3-VL-30B-A3B. ~68 tok/s on M4 Max, 32 GB minimum. The MLX conversion handles continuous batching for vision through vllm-mlx if you need it. The full open-weights landscape, including Qwen, Llama, DeepSeek, Mistral, Gemma, and the rest, is mapped in our [open-source LLMs landscape pillar](https://codersera.com/blog/open-source-llms-landscape-2026/). The infrastructure side — cloud GPU runners, Kubernetes deployment, multi-tenant serving — is in our [self-hosting LLMs guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/). ## How should I think about quantization on Mac? Quantization compresses weights from 16-bit floats down to lower-bit representations, trading some accuracy for huge memory savings. The two formats that matter on Mac: **MLX native quants** (`-4bit`, `-8bit`, mixed precision). The default. `mlx_lm.convert --hf-path X --q-bits 4` converts any Hugging Face model. Quality at 4-bit on 7B+ models is within 1–2% of bf16 on standard benchmarks. Sub-3B models suffer noticeably at 4-bit — prefer 8-bit there. **GGUF** (Q2_K through Q8_0, plus K-quants and IQ-quants). The cross-platform standard, used by llama.cpp, Ollama (the legacy backend), and LM Studio. Q4_K_M is the safe default for 13B+ models — within ~1% of FP16 on quality. Q5_K_M for 7B and smaller. Q8_0 if you have the RAM headroom and want near-lossless. Below Q4_K_M, use importance-matrix quantization (imatrix) variants, otherwise quality drops noticeably. What to *skip* on a Mac: AWQ, GPTQ, EXL2. All three are NVIDIA tensor-core-optimised; no Mac tool supports them natively. The few "AWQ on Mac" claims you'll see online are running the underlying base weights through a conversion, not the AWQ format itself. The general rule for Mac users in 2026: **start at MLX 4-bit**. Move to 8-bit if your model is under 7B or you notice quality issues. Move to bf16 if you have memory headroom and care about long-form reasoning quality. Only reach for GGUF when the model you want hasn't been converted to MLX yet (the gap is usually a few days). ## Can I run frontier models by clustering multiple Macs? Yes, and this is the most interesting 2026 development for sovereign-AI teams. The pattern: connect 2–4 Macs over Thunderbolt 5, install MLX, run distributed inference across the cluster. A small group of Mac Studio Ultras now serves DeepSeek V4 at higher precision than any single machine could fit. The enabling technology is **JACCL** — a distributed backend Apple shipped with macOS 26.2 that runs MLX collectives over RDMA on Thunderbolt 5, hitting 50–60 Gbps with sub-50 µs latency. An order of magnitude lower latency than the previous ring-allreduce backend. Requires a fully-connected Thunderbolt topology (every Mac directly cabled to every other Mac), which limits practical cluster sizes to 4–6 nodes. The community tool that wraps this is **EXO**, which auto-discovers Macs on the local network and distributes a single model across them, transparently. Spin up two Mac Studio Ultras, point EXO at them, and you have a single-binary serving endpoint for a 400B+ MoE that wouldn't fit on either individually. When clustering is worth it: you're a small team, you need data sovereignty (the entire model and prompt history never leaves your office), you'd otherwise be on a $50K/mo OpenAI bill, and you can amortise the hardware (~$30K–$60K depending on configuration) over a year. When it's not worth it: you're a solo developer; rent an H100 on Lambda for $2.50/hour instead. ## What about Apple's own on-device LLM (Foundation Models)? In WWDC 2025, Apple shipped the **Foundation Models framework** in iOS / iPadOS / macOS 26 — a ~3 B-parameter on-device LLM with a Swift API. It powers Apple Intelligence behind the scenes. From a developer's perspective, you call a Swift function, you get tool-using LLM output, and nothing leaves the device. Strengths: free, offline, private, optimised for Apple Silicon, no install. Weaknesses: closed weights, Swift-only, fixed size, modest quality (3 B parameters tuned for on-device, not for hard reasoning). It's not a replacement for an MLX-served open-weight model — it's a complementary option when you ship a Mac/iOS app and want a default model that just works. WWDC 2026 (June 8–12) is widely expected to expand the framework — bigger on-device models, better Swift / Xcode integration, possibly a "Core AI" replacement for Core ML. If that ships, an Apple Foundation Models pillar is the natural follow-up to this guide. ## Can I fine-tune on my Mac? Yes, with caveats. MLX-LM ships LoRA, QLoRA, and DoRA fine-tuning natively, with realistic memory footprints: - **7–8 B model:** QLoRA at 4-bit fits in 7–8 GB working memory — runs on a 16 GB MacBook Pro - **13–14 B:** 14–18 GB working memory, comfortable on 32 GB - **32 B at QLoRA:** ~20–25 GB, possible on 48 GB+ - **70 B:** requires 96 GB+ unified memory; tight but works Real-world example: a Mistral 7B QLoRA run on 5,000 examples takes ~90 minutes on an M2 Max 32 GB, peak ~7 GB RAM. The unified-memory trick again — a 32 GB Mac fine-tunes models that OOM an RTX 3090's 24 GB VRAM. The tradeoff: NVIDIA still trains 2–4× faster on whatever fits on it, so for serious multi-day fine-tuning work, rent a cloud H100. The CLI is dead simple: `mlx_lm.lora --model mlx-community/Llama-3.1-8B-Instruct-4bit --train --data ./data --iters 1000`. Our [fine-tuning LLMs guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) covers the practical recipe — dataset format, LoRA hyperparameters, evaluation — in depth. ## How do I get started in five minutes? **Path A — Ollama (recommended for most users).** 1. Install: `brew install ollama` then `ollama serve &` 2. Pull a model: `ollama pull qwen3-coder:30b` 3. Chat: `ollama run qwen3-coder:30b` 4. For programmatic use, hit the OpenAI-compatible endpoint at `http://localhost:11434/v1` **Path B — MLX-LM directly (recommended for max performance and fine-tuning).** 1. Install: `pip install mlx-lm` 2. Generate: `mlx_lm.generate --model mlx-community/Qwen3-Coder-30B-A3B-4bit --prompt "Write a Python script that..."` 3. Serve an OpenAI-compatible API: `mlx_lm.server --model mlx-community/Qwen3-Coder-30B-A3B-4bit` 4. Fine-tune: `mlx_lm.lora --model <model> --train --data --iters 1000` **Path C — LM Studio (recommended for GUI users).** Download from [lmstudio.ai](https://lmstudio.ai), hit the model marketplace, filter by "MLX" or "Apple Silicon", click download, click load, chat. Includes an OpenAI-compatible local server you can flip on with one click. ## FAQ ### Is MLX faster than llama.cpp on Mac? Yes, in 2026, by 30–60% on most workloads on M4 and M5 hardware. The gap is widest on prompt processing (time-to-first-token), where MLX uses Apple's Neural Accelerators that llama.cpp doesn't. On older M1 and M2 chips the gap is smaller. For any new Mac you buy in 2026, default to MLX-backed tooling (Ollama 0.19+, MLX-LM, vllm-mlx, or LM Studio's MLX backend). ### How much RAM do I really need for serious local-LLM work? 32 GB is the entry point for genuinely useful work — it runs the 30B-A3B MoE class at ~100 tok/s, which feels like a cloud API. 64 GB is the sweet spot — adds 70B dense models and gives breathing room for context. 96–128 GB is for power users who want bf16 70B or larger MoEs. Anything below 32 GB confines you to small models that are good for narrow tasks but won't replace a hosted API. ### Should I buy a MacBook or a Mac Studio for local LLMs? MacBook Pro 14"/16" with M4 Max 64 GB if you need portability — it's the best laptop money can buy for LLM work. Mac Studio M4 Max 64 GB if you want a desk machine at lower cost. Mac Studio M2/M3 Ultra 192 GB only if you need to run 100B+ models locally; the cost-per-token is much better than buying a discrete NVIDIA setup at the same scale, but you're paying for the unified memory pool. ### Can I run Claude or GPT on my Mac? No — neither Anthropic nor OpenAI release weights, so their models can only be accessed via their respective APIs. The best *open-weight* alternatives in May 2026 are DeepSeek V4 (rivals GPT-5.5 on many benchmarks), Kimi K2.6 (rivals Claude Opus 4.6 on agentic tasks), and Qwen 3.6 / 3.5 (the strongest all-around open-weights family). For an honest comparison see our [open-source LLMs landscape pillar](https://codersera.com/blog/open-source-llms-landscape-2026/). ### What's the difference between MLX 4-bit and GGUF Q4? Both compress a 16-bit model to roughly 4 bits per weight. MLX's 4-bit format is optimised for Apple's GPU shaders and runs faster on Apple Silicon. GGUF Q4_K_M is portable across platforms. Quality is comparable for 7B+ models. Pick MLX if the model has been converted (most have); pick GGUF only when the model is too new to have an MLX variant. ### Does the M4 vs M5 difference matter for LLMs? Yes, for prefill speed. M5 is dramatically faster at processing long prompts (3–4× faster time-to-first-token in Apple's published benchmarks) because of the Neural Accelerators. Token-by-token generation is only ~15% faster because that's memory-bandwidth-bound and bandwidth only grew modestly. If you do a lot of long-context work (RAG, agent loops, code repos), M5 is worth the upgrade. If you do short chat, the difference is smaller. ### Can I use a Mac as the primary serving box for a SaaS app? Honest answer: usually no. A Mac handles 1–10 concurrent users gracefully via vllm-mlx; beyond that you want a cloud GPU. The Mac is excellent for internal tools, agent prototypes, dogfooding, and small-team deployments. For production serving at scale, see our [self-hosting guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) for the cloud/Kubernetes path. ### What about training a model from scratch on my Mac? Not realistic at meaningful scale — you'd need weeks to months of GPU time even for a small model, and a Mac Studio Ultra doesn't beat a single rented H100 for raw FLOPS. The right path for "I want to train a model" is cloud H100/H200 spot instances. See our [guide on self-training small LLMs](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) for the realistic recipe — what fits, what costs what, and when nanochat is the right starting point. ### How do I keep MLX up to date? `pip install --upgrade mlx mlx-lm` once a month is enough. Ollama updates itself when you run `brew upgrade ollama`; LM Studio prompts you in-app. The model side moves faster than the framework side — check `mlx-community` on Hugging Face weekly for new conversions. ### What's missing or coming next? WWDC 2026 (June 8–12) is the next big inflection point. Expected: Foundation Models framework v2, deeper Xcode integration ("Xcode Intelligence"), possibly a Core AI replacement for Core ML, and likely larger on-device models. The macOS 27 release in autumn 2026 will probably ship more JACCL improvements for clusters. On the open-weights side, expect Llama 5 (probably 2027 per Meta), Qwen 4, and continued MoE growth. ## Related guides - [Self-hosting LLMs in the cloud](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) — the Linux + Kubernetes + multi-GPU side of the same coin - [Fine-tuning LLMs](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) — practical recipes for LoRA / QLoRA / DoRA / MLX-LoRA / Unsloth / Axolotl - [Self-training a small LLM from scratch](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) — when pre-training a model yourself actually makes sense - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) — the full family of open-weight models in 2026 - [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) - [Qwen 3.5 complete guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) - [Llama 4 complete guide](https://codersera.com/blog/llama-4-complete-guide-2026/) - [Gemma 4 complete guide](https://codersera.com/blog/gemma-4-complete-guide-2026/) - [Kimi K2.6 complete guide](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/) - [AI coding agents (Claude Code, Cursor, Copilot)](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) =============================================================== [15/20] Fine-Tuning LLMs: The Complete 2026 Guide (LoRA, QLoRA, Unsloth, Axolotl, MLX) URL: https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer.** Fine-tuning an open-weight LLM in May 2026 is a one-command operation. Pick QLoRA (4-bit base + LoRA adapter) with rank 16 on top of Llama 3, Qwen 3, Gemma 4, or Mistral. Use **Unsloth** on a single GPU (free tier handles 7B; a rented H100 handles 70B in under three hours), **Axolotl** for multi-GPU production, **MLX-LoRA** on Mac, or **TRL** when you need raw control. Most jobs need 500–2,000 hand-curated examples in ChatML format and run in under an hour. But the most important step is upstream: *don't fine-tune for knowledge* (use RAG), *don't fine-tune for format* (use structured outputs), and don't fine-tune until prompt engineering has stopped paying off. When you do fine-tune, the right reasons are narrow style / niche reasoning / cheaper inference via distillation. This guide covers the 2026 framework landscape, the decision tree, dataset prep, hyperparameters, evaluation, and the failure modes that quietly ruin most first runs. ## Should you fine-tune at all? Fine-tuning has a reputation as the "advanced" answer to LLM problems. In 2026 it is rarely the right first move. Three patterns reliably beat fine-tuning on the most common requests: - **RAG (retrieval-augmented generation)** is the right answer when you want the model to know *more facts* — internal docs, current events, private data, customer history. Fine-tuning baked-in knowledge ages badly (the data is stale the moment you ship) and can be wrong in ways that are hard to detect. RAG keeps the knowledge in a vector store you can update independently. - **Structured outputs / tool calling** is the right answer when you want a specific format — JSON, function call, schema-validated response. Modern model APIs and open-weight chat templates support strict JSON / Pydantic schemas; you don't need a fine-tune to get reliable structure. - **Prompt engineering with in-context examples** handles most narrow tasks if you can fit 5–10 examples in the prompt. Frontier models in 2026 have 128 K–1 M context windows; you can include a small "training set" inline and skip the fine-tune entirely. The cases where fine-tuning genuinely wins: 1. **Brand voice / style.** A 500–2,000 example QLoRA on Llama 3 8B reliably teaches a model to write in your house tone, faster and cheaper than a 10 K-token system prompt. 2. **Niche reasoning style.** Legal citations, medical SOAP notes, scientific paper sections — domain conventions that prompt engineering can describe but not reliably enforce. 3. **Distillation for inference cost.** Use Claude Opus 4.7 or GPT-5.5 to generate 50–100K high-quality examples, then fine-tune Llama 3 7B or Qwen 3.6 8B to mimic the behaviour on your narrow task. You'll serve at 1/100th the cost. 4. **Tool-calling fine-tunes** where the model needs to learn a custom function schema reliably. 5. **Multilingual or rare-language work** where the base model is weak. If your use case isn't on that list, finish prompt engineering and RAG first. Then come back. ## LoRA, QLoRA, DoRA: the parameter-efficient core Every fine-tune you'll do in 2026 uses one of these adapter techniques. They freeze the base model's billions of weights and train tiny matrices on top, capturing the "task delta" without touching the foundation. **LoRA (Low-Rank Adaptation).** Updates two small matrices *A* and *B* whose product approximates a weight delta. Two hyperparameters: `r` (rank — the bottleneck width) and `alpha` (the scale factor). A LoRA adapter is typically a few hundred MB instead of the original model's tens of GB. **QLoRA.** Keeps the base model in 4-bit NF4 quantization and trains an fp16 LoRA on top. The memory savings are dramatic — a 70B model fits in ~48 GB of VRAM instead of ~140 GB. Quality is within 1–2% of full LoRA on standard benchmarks. **This is the 2026 default.** **DoRA (Weight-Decomposed LoRA).** Decomposes the weight update into magnitude and direction, then applies LoRA only to the direction. Converges faster and often matches full fine-tuning at the same rank. Frameworks default DoRA on in 2026; `use_dora=True` is a free upgrade. **Rank and alpha defaults that work:** - Style / voice tasks: `r=16, alpha=16` - General SFT: `r=32, alpha=32` - Complex multi-turn or code: `r=64, alpha=64` Target all linear layers (`q,k,v,o,gate,up,down`). The VRAM cost of targeting all-linear vs just q/v is small and the quality gain is consistent. Ignore the old `alpha=2*r` convention; Unsloth's 2026 ablations show `alpha=r` is the cleaner default. ## Which framework should I use? Pick by the shape of your hardware, not by which library is trending on X. | Framework | Right when | Wrong when | | --- | --- | --- | | **Unsloth** | Single GPU (RTX 3090/4090/5090 or one rented H100). Indie hacker / solo dev / startup. You want fastest training in the cleanest notebook. | Multi-GPU (open-source is single-GPU only; multi-GPU costs $10/mo). | | **Axolotl** | Multi-GPU, long context (sequence parallelism), CI-integrated training, you want YAML configs you can diff in PRs. | Single GPU — Unsloth is faster. | | **HF TRL + PEFT** | You need a custom training loop, custom reward model, research, or you're building your own training framework. | Hello-world fine-tunes — too much boilerplate. | | **MLX-LM (Mac)** | Apple Silicon. Up to 70B QLoRA on 96 GB Mac Studio Ultra. Quiet, no fan noise, no cloud bill. | Serious multi-day runs; NVIDIA still trains 2–4× faster on whatever fits in VRAM. | | **LLaMA-Factory** | You want a web GUI for fine-tuning, multi-model + multi-method support, no code. | Production reproducibility (the YAML/CLI path in Axolotl is cleaner). | **Unsloth** shipped MoE support in Feb 2026 with a claimed 7–12× speedup on MoE fine-tuning, runs 500+ base models including Llama 4, Qwen 3.6, Gemma 4, DeepSeek V4, and the gpt-oss family. A 70B QLoRA fits and trains in ~2.8 hours on a single H100. Open-source Unsloth is single-GPU only; multi-GPU is Unsloth Pro ($9.99/mo) — for a solo dev this is the right tradeoff. **Axolotl v0.8.x** is the production maturity point — config + `accelerate launch -m axolotl.cli.train config.yml` is a complete pipeline. Supports SFT, LoRA, QLoRA, DPO, KTO, ORPO, GRPO, full reward modelling, quantization-aware training, and recently shipped sequence parallelism for >128 K context training. Slower per-GPU than Unsloth on single-GPU runs, but multi-GPU scaling is real and the operational story is much cleaner. **HF TRL** hit v1.0 in April 2026 and unified the post-training stack: SFTTrainer, DPOTrainer, KTOTrainer, ORPOTrainer, GRPOTrainer, RewardTrainer in one library. v1.0 even pulls in Unsloth kernels for a 2× SFT speedup. Reach for it when you want full control of the loop. **MLX-LM** on Mac handles LoRA, QLoRA, and DoRA via `mlx_lm.lora --train`. Realistic ceiling: 32 GB Mac fine-tunes 7–8B comfortably; 96 GB Mac Studio Ultra fine-tunes 70B QLoRA. The unified-memory advantage means a 32 GB Mac fine-tunes models that OOM a 24 GB RTX 3090. See our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) for the full picture. ## What about hosted fine-tuning APIs? The hosted landscape changed in early 2026: **OpenAI.** Wound down fine-tuning for new models in May 2026. Fine-tuning still works on the GPT-4.1 family and o4-mini for existing customers, but no GPT-5 / 5.4 / 5.5 fine-tunes. OpenAI's bet is that prompts + tools + memory beat bespoke weights for most use cases. Use the legacy fine-tune only if you specifically need a closed-weight API model with API-grade reliability and your data volume is too small to justify any infra (<5K examples). **Anthropic.** Claude fine-tuning is *not* generally available. Older Claude models (Haiku tier) can be fine-tuned through AWS Bedrock or Google Vertex AI under a managed-service workflow. Opus 4.7, Sonnet 4.6, and the current frontier tier are **not fine-tunable** — Anthropic positions prompt caching, extended thinking, and tool use as the right knobs. If you need Claude-quality behaviour in a fine-tune in 2026, you can't have it — distil to Llama 3 or Qwen 3.6 instead. **Together AI.** Every major Llama / Mistral / Qwen size up to 405B. LoRA at ~$0.48 per million training tokens (≤16B), full fine-tune at ~$3.20 per million tokens (70–100B). Fine-tuned adapters serve at base-model inference price plus a small overhead. **The cheapest hosted path for most teams.** **Fireworks AI.** Similar pricing to Together. Strong DPO support at 2× SFT cost. Fine-tuned models serve at base price (no surcharge). The right pick when you want RLHF-style alignment without infrastructure. **Lamini.** $0.50 per million inference tokens plus $0.50 per tuning step, with $300 free credit. Niche around their "Memory Tuning" (Mixture of Memory Experts) for factual-recall use cases where hallucination is the failure mode. Most teams don't need this; Together / Fireworks is the safer default. Rule of thumb: API-hosted wins when you'll serve under 1 M tokens/day, when you don't want to manage GPUs, or when you need an SLA you can't personally provide. Local fine-tuning wins on cost above that threshold, on privacy, and on iteration speed. ## How do I prepare a fine-tuning dataset? **Format.** ChatML (`<|im_start|>role\ncontent<|im_end|>`) is the 2026 default everywhere — OpenAI, Bedrock, Vertex, every major open-weight chat model. Use it. ShareGPT (turns array with `from`/`value`) is fine for multi-turn community datasets. Alpaca (`instruction`/`input`/`output`) is the simplest format for single-turn classification or extraction. **Pick one and be consistent.** Template inconsistency is the single biggest cause of silent fine-tune failures. **Size.** 500–1,000 clean examples genuinely works for narrow tasks (classification, format conversion, brand voice). 5,000–10,000 for general SFT. **Quality beats quantity by a wide margin.** 500 hand-curated examples consistently outperform 5,000 LLM-scraped ones with format drift and quality variance. **Hygiene.** Deduplicate (MinHash or simple exact-match), length-filter, run a quick contamination check against the eval set you'll use, validate the chat template renders correctly on 3–5 random samples before launching a multi-hour run. The `datatrove` library from Hugging Face handles dedup at scale; for a small dataset, a Python one-liner over hashes is enough. **Contamination check.** If your eval data was in the base model's pretraining corpus, your "improvement" is just recall. Always hold out a fresh eval set written or collected *after* the base model's training cutoff. A 2026 Berkeley study found 8 major agent benchmarks could be exploited to near-perfect scores without solving anything, via leaked references and broken scoring. Don't trust any benchmark you didn't read the source of. ## What hyperparameters should I use? The cheat sheet that works across most chat-style QLoRA fine-tunes: - **Learning rate:** 1e-4 to 2e-4 for LoRA on 7B–14B; 5e-5 to 1e-4 for full fine-tune; 1e-5 to 5e-5 if you see catastrophic forgetting. - **Batch size:** Effective batch of 8–32 sequences. Use gradient accumulation freely — physical batch can be as low as 1 with accumulation of 16. - **Epochs:** 1–3. More than 3 on a small dataset overfits. - **Sequence length:** Set to the 95th percentile of your dataset's tokenized length. Padding to the max wastes compute. - **Warmup:** 3–10% of total steps. Cosine decay schedule. - **Optimizer:** 8-bit AdamW (saves VRAM, minimal quality cost). - **LoRA dropout:** 0.05 is a safe default; raise to 0.1 if overfitting. Track training loss *and* validation loss every N steps; stop when validation plateaus. The most common amateur mistake is "train for 3 epochs because the docs said so" — early-stop on the validation curve. ## How do I evaluate a fine-tune? Three layers, in this order: **1. General capability regression.** Run `lm-eval-harness` (EleutherAI) on MMLU / ARC / HellaSwag / GSM8K before and after to confirm you didn't break the base. A 1–3% drop on general benchmarks is normal; 10%+ is catastrophic forgetting. **2. Task-specific eval.** Write a custom eval — 100–500 held-out examples scored by exact-match (for structured outputs) or a frontier-model judge (for generative quality). The judge pattern is standard in 2026: ask Claude Opus 4.7 or GPT-5.5 to grade outputs against a rubric. Cheap, fast, and surprisingly reliable if the rubric is clear. **3. Production A/B test.** The only eval that can't be gamed. Ship to 5–10% of traffic, log outputs, compare against the base on real workloads. Anything that passes layers 1 and 2 but fails layer 3 was suffering from train/test distribution shift you didn't catch. ## Common failure modes **Catastrophic forgetting.** Fine-tune on narrow data → model loses general capability. Mitigations: lower learning rate, train fewer epochs, mix in 10–20% general-instruction data (the *regularization* approach), prefer LoRA over full fine-tune (the base stays frozen, so forgetting is bounded), or use DoRA which empirically retains better. **Overfitting tiny datasets.** 3 epochs on 500 examples and the model memorizes them. Always hold out 5–10% as validation; early-stop on plateau. **Base-model contamination.** If your eval data was in the base's pretraining corpus, your "improvement" is just recall. Hold out fresh data written after the base's cutoff. **Template drift.** Training on ChatML, inferencing on Alpaca → nonsense outputs. Always use the same chat template at inference that you trained with. `tokenizer.apply_chat_template` is your friend. **Silent format failures.** The model trains successfully but the dataset's role tags were wrong, so it learned nothing useful. *Always* render 5 random training examples with the tokenizer's chat template and eyeball them before launching the run. ## How much does fine-tuning actually cost? Realistic 2026 cost ranges on cloud spot pricing: | Model | Method | Dataset | Hardware | Wall clock | Cost | | --- | --- | --- | --- | --- | --- | | Llama 3 7B | QLoRA r=16 | 5K examples | 1× T4 (Colab free) | ~6h | $0 | | Llama 3 7B | QLoRA r=16 | 5K examples | 1× A100 40GB | ~45 min | ~$1.50 | | Qwen 3.5 14B | QLoRA r=32 | 10K examples | 1× A100 80GB | ~2h | ~$3.50 | | Llama 3 70B | QLoRA r=64 | 10K examples | 1× H100 80GB | ~3h | ~$8 | | Mistral 7B | QLoRA on Mac | 5K examples | M2 Max 32GB | ~90 min | $0 (own hardware) | | Llama 4 70B | Full LoRA | 50K examples | 8× H100 | ~12h | ~$320 | The CoreWorxLab "fine-tune Qwen 3.5 for $11 on a rented A100" YouTube clip floating around in April 2026 is real; the workflow is plausibly Unsloth's Qwen 3.5 Colab template on a single A100. Anyone telling you fine-tuning is expensive is talking about full pre-training, not LoRA. ## Quick start: three concrete recipes **Recipe A — Llama 3 7B QLoRA on free Colab, Unsloth.** ``` !pip install unsloth from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="unsloth/llama-3-8b-bnb-4bit", max_seq_length=2048, load_in_4bit=True) model = FastLanguageModel.get_peft_model( model, r=16, lora_alpha=16, use_dora=True, target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"]) # ...load dataset, train with SFTTrainer, save adapter ``` **Recipe B — Mistral 7B QLoRA on an M-series Mac, MLX-LoRA.** ``` pip install mlx-lm mlx_lm.lora \ --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \ --train \ --data ./my-jsonl-dir \ --iters 1000 \ --batch-size 4 \ --lora-layers 16 \ --save-every 200 \ --adapter-path ./adapters ``` **Recipe C — Multi-GPU production fine-tune, Axolotl.** ``` # config.yml base_model: meta-llama/Llama-3-70B adapter: qlora load_in_4bit: true sequence_len: 4096 lora_r: 64 lora_alpha: 64 lora_dropout: 0.05 lora_target_linear: true use_dora: true datasets: - path: ./train.jsonl type: chat_template num_epochs: 2 learning_rate: 1e-4 gradient_accumulation_steps: 4 micro_batch_size: 1 warmup_steps: 100 lr_scheduler: cosine optimizer: adamw_bnb_8bit # then: accelerate launch -m axolotl.cli.train config.yml ``` ## FAQ ### What's the difference between LoRA, QLoRA, and DoRA? LoRA trains tiny adapter matrices on top of a frozen base model — much cheaper than full fine-tuning. QLoRA adds 4-bit quantization of the base, cutting memory roughly 4× with negligible quality loss. DoRA decomposes the weight update into magnitude and direction, often matching full fine-tune quality at the same rank. Default in 2026: QLoRA + DoRA together. ### How much data do I need to fine-tune? 500–2,000 examples for narrow tasks like style, voice, classification, or format conversion. 5,000–10,000 for broader instruction tuning. Quality dominates quantity: 500 hand-curated examples beat 5,000 LLM-scraped ones almost every time. ### Should I fine-tune Llama 3, Qwen 3.6, Gemma 4, or Mistral? For most general tasks in 2026: Qwen 3.6 8B or Llama 3 8B. For tiny on-device deployment: Gemma 4 E2B. For coding specifically: Qwen3-Coder. For multilingual: Qwen 3.6. See our [open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) for the full comparison. ### Can I fine-tune Claude or GPT? Not on the current frontier models. OpenAI offers fine-tuning on legacy GPT-4.1 and o4-mini. Anthropic offers Claude Haiku fine-tuning through Bedrock and Vertex but not Sonnet/Opus. The 2026 reality is: if you need Claude or GPT-class quality on a fine-tune, you can't have it directly — distil their outputs to an open-weight model instead. ### What's the cheapest way to fine-tune? Colab free tier with Unsloth for 7B QLoRA — literally $0 if it fits in your patience window. Rented H100 spot on Lambda or RunPod at $2.50/hr for serious work — most 7B–14B fine-tunes finish in under an hour. Together AI's hosted fine-tuning at ~$0.48 per million training tokens if you don't want to touch GPUs at all. ### How long does a typical fine-tune take? A 7B QLoRA on 5K examples: ~45 min on an A100, ~6h on Colab free T4. A 70B QLoRA on 10K examples: ~3h on an H100. A full multi-day pre-training run is a different problem entirely — see our [self-training guide](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/). ### When is RAG better than fine-tuning? Always, when the goal is to give the model access to *knowledge*. RAG keeps data fresh, auditable, and updatable independently of the model. Fine-tuning baked-in facts is brittle: the data stales, hallucinations are harder to detect, and updating means a new training run. Use fine-tuning for *behaviour* — style, format, reasoning patterns — and RAG for *facts*. ### What about catastrophic forgetting? Real and common. The model "forgets" general capabilities while specialising. Mitigations: lower learning rate (1e-5 to 5e-5), train fewer epochs (often 1 is enough), mix 10–20% general instruction data into your training set, prefer LoRA over full fine-tune (the base stays intact under the adapter), and use DoRA which retains better empirically. ### Can I run a fine-tuned model on a Mac? Yes. Train on cloud H100 if needed, then download the adapter and merge it with the base, convert to MLX 4-bit, and run via `mlx_lm.generate` or Ollama. The whole serving side is covered in our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/). ### What about DPO, ORPO, GRPO, SimPO? Post-SFT alignment techniques for preference tuning. DPO is the foundational pairwise-preference method. ORPO merges SFT and DPO into one stage — the simplest pipeline. GRPO is what DeepSeek R1 popularised for reasoning-with-verifiable-rewards (math, code). For most fine-tunes: SFT alone is enough. Add ORPO if you have preference pairs. Reach for GRPO only if you're going after a reasoning benchmark specifically. ### How do I deploy a fine-tuned model? Three options. (1) Self-host: convert adapter to MLX or GGUF, serve via Ollama / MLX-LM / vLLM. (2) Hosted: upload the adapter to Together / Fireworks, serve at base-model price. (3) Edge: quantize to 4-bit and run on-device. The right path depends on traffic volume and SLA needs — see our [self-hosting guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/). ## Related guides - [Apple Silicon LLMs](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) — running and fine-tuning models on Mac with MLX - [Self-training a small LLM from scratch](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) — when pre-training a model yourself actually makes sense - [Self-hosting LLMs](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) — serving infrastructure in the cloud - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) — which base model to fine-tune - [Llama 4 guide](https://codersera.com/blog/llama-4-complete-guide-2026/) - [Qwen 3.5 guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) - [Gemma 4 guide](https://codersera.com/blog/gemma-4-complete-guide-2026/) - [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) =============================================================== [16/20] Self-Training a Small LLM From Scratch: The 2026 Complete Guide URL: https://codersera.com/blog/self-training-small-llm-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer.** Pre-training an LLM from scratch in 2026 is dramatically cheaper than it was in 2024 — you can train a GPT-2-quality model end-to-end for around **$50 on an 8× H100 spot instance** using [Karpathy's nanochat](https://github.com/karpathy/nanochat) — but for almost every product use case the right answer is still to fine-tune an existing open-weight model, not pre-train. Pre-training makes sense when (a) you have a genuinely novel domain (rare language, niche scientific corpus, proprietary code) with 10B+ clean tokens no open model has seen, (b) you have $20K+ in compute budget you've already exhausted on fine-tuning, or (c) you want to learn how the stack works. This guide covers the realistic recipes — nanochat for learning, modded-nanoGPT for speedrunning, lit-gpt for production — the modern 2026 architecture template (decoder-only + RoPE + RMSNorm + SwiGLU + GQA), the open datasets that actually work (FineWeb, FineWeb-Edu, SlimPajama, The Stack v2), honest cost estimates across model sizes, and a decision tree for when self-training beats fine-tuning. Reading it carefully will save most teams from a six-figure mistake. ## Should you train a model from scratch? For 99% of teams, no. Pre-training a model means building from zero what Meta, OpenAI, and Anthropic spend tens of millions of dollars on every few months. Even with 2026's cost reductions, a Llama-3-class 8B model trained at the now-standard "trillions of tokens" scale costs **$1M–2M** on spot pricing. A Chinchilla-optimal 7B at 140B tokens is ~$30K on spot. A tiny "learning project" 700M model trained to GPT-2 quality is genuinely $50. The handful of cases where pre-training actually wins: - **Genuinely novel domain.** Medical, legal, code in a niche language, a rare natural language, a closed scientific corpus the open models haven't seen. If your tokens are meaningfully different from web text, an existing tokenizer will be inefficient and an existing model will hallucinate where it should defer. - **Data sovereignty.** Regulated industries where you can't use a model trained on unknown web data — defence, certain healthcare deployments, classified work. - **Edge-deployment niche.** A 150M–500M model purpose-built for your task and quantised to 4-bit can run on a phone CPU. Llama 3 1B is overkill for most edge use cases. A custom small model can be 3–10× more efficient on your domain. - **Learning.** Understanding the full stack — tokeniser, dataloader, optimiser, distributed training, eval — is the highest-leverage skill in AI engineering. [Karpathy's nanochat](https://github.com/karpathy/nanochat) exists for exactly this. It's the best 4 hours you can spend. For everything else: [fine-tune an open-weight model](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) instead. Better quality, faster, cheaper, lower risk. ## What does "small LLM" actually mean in 2026? The 2026 small-LLM bands and what they cost to train, on cloud spot pricing (H100 at ~$2.50/hr, 8× node at ~$18/hr): | Target | Tokens | GPU-hours (H100 80GB) | Spot cost | Real-world example | | --- | --- | --- | --- | --- | | 150M, 30B tokens | 30B | ~150 | ~$200 | Tiny edge model | | 700M, 40B tokens | 40B | ~30 on 8× H100 | ~$50 | nanochat speedrun | | 1B, 50B tokens | 50B | ~600 | ~$1.5K | Chinchilla-optimal 1B | | 1B, 3T tokens | 3T | ~16,000 | ~$10K–15K | TinyLlama scale (inference-optimal) | | 3B, 60B tokens | 60B | ~5,500 | ~$14K | Chinchilla-optimal 3B | | 7B, 140B tokens | 140B | ~30,000 | ~$30K–50K | Chinchilla-optimal 7B | | 7B, 15T tokens | 15T | ~3,000,000 | ~$1M–2M | Llama-3-class quality | H200 is ~1.3× faster than H100, B200 is ~2–3× — those numbers shift downward 30–60% on newer hardware. The dramatic point is that **"GPT-2 quality" is now a $50 weekend project**; the dramatic point in the *other* direction is that **Llama 3 quality is a $1M+ training run** even at spot. ## Has Chinchilla's "20 tokens per parameter" rule broken? Yes, in the inference-optimal direction. Chinchilla (2022) defined compute-optimal training: train a 1B model on ~20B tokens, a 70B on ~1.4T. In 2024–2026, the field shifted to **inference-optimal** training — train far past Chinchilla because the per-token inference savings dwarf the extra training cost when you're going to serve the model for years. Llama 3 8B trained at ~200:1 tokens-per-parameter (15T tokens for 8B). Liquid AI's LFM2.5-350M hit 80,000:1 (28T tokens on 350M params) in April 2026. The general 2026 rule for any model you'll actually serve: **train at least 100:1, ideally 500–1,000:1 for small models**. For a $50 learning project, Chinchilla is fine; for a production model you'll embed in an app, push way past. ## What's the modern 2026 architecture? The small-LLM architecture is essentially solved. Every open-weight model that matters in 2026 — Llama 3, Qwen 3.6, Gemma 4, Mistral, DeepSeek V4, Granite 4 — converged on the same template. Copy it; don't innovate: - **Decoder-only transformer.** Encoder-decoder is dead for language; sparse-attention / SSM hybrids are experimental. - **Pre-norm with RMSNorm.** LayerNorm is legacy. RMSNorm trains more stably and slightly faster. - **Rotary positional embeddings (RoPE).** With YaRN or similar scaling for context-length extension. - **SwiGLU FFN.** Gated linear unit with Swish activation. Replaced ReLU/GELU FFNs. - **Grouped-query attention (GQA).** Typically 4–8× fewer KV heads than Q heads. Cuts KV-cache memory dramatically with negligible quality loss. - **No bias terms.** In Q/K/V projections and FFN, bias-free is standard. - **Tied embeddings** for small models (output projection shares weights with input embedding). Save parameters that aren't pulling weight. If you want to experiment beyond this: MoE for parameter efficiency at the cost of training complexity, sliding-window attention for cheap long context, or Mamba/SSM hybrids for very long sequences. Otherwise: copy a Llama-3 config, scale dimensions to your budget, move on. ## Which reference recipe should I use? Three project shapes, three right answers: **1. Learning / "I want to understand the stack" —**[**karpathy/nanochat**](https://github.com/karpathy/nanochat)**.** The canonical solo-dev starting point. $48 on an 8× H100 node (~$15 on spot), ~1.65 hours wall clock as of March 2026. Single `speedrun.sh` script takes you tokenizer → pretrain → mid-train → SFT → optional RL on GSM8K → eval → web chat UI. The code is short enough to read end-to-end in an afternoon. **If you're new to pre-training, start here.** **2. Speed-record geeking —**[**KellerJordan/modded-nanogpt**](https://github.com/KellerJordan/modded-nanogpt)**.** The community speedrun fork of nanoGPT. As of April 2026, the record is *1.35 minutes to GPT-2-quality on 8× H100*, achieved through the Muon optimiser, Flash Attention 3, FP8 head, learnable cross-stream attention, and multi-token prediction. Reading the commit history is a graduate course in modern training tricks. Use this when you want to learn what's actually fast in 2026. **3. Production-grade work —**[**Lightning-AI/litgpt**](https://github.com/Lightning-AI/litgpt)**.** 20+ supported architectures, production-grade pretrain + fine-tune + deploy. Powered the original TinyLlama. Use this when nanochat's hackability stops being a feature and you need a stable CLI, FSDP2 / TP scaling, and someone-else-can-inherit-this configuration. Pair with **torchtitan** for multi-node training beyond a single 8× H100 box. Other notable references worth reading: - [EleutherAI Pythia](https://huggingface.co/EleutherAI/pythia-1.4b) — 160M to 12B series with intermediate checkpoints, the gold-standard "training dynamics" reference - [Allen AI OLMo](https://allenai.org/olmo) — fully open weights + data + intermediate states - [TinyLlama](https://github.com/jzhang38/TinyLlama) — the canonical 1.1B model trained on 3T tokens, the inference-optimal pattern ## What datasets should I use? For general pre-training, three open corpora dominate 2026: **FineWeb** (Hugging Face, 15T tokens). Common Crawl with careful filtering. The default open base. Outperforms RedPajama V1 and RefinedWeb in head-to-head ablations. **FineWeb-Edu** (1.3T tokens). Educational-content-filtered subset of FineWeb. Dramatically better MMLU / ARC numbers per training-token than vanilla FineWeb. Use when knowledge density matters for the downstream tasks. **RedPajama V2** (100T+ tokens with quality metadata). Useful when you want to apply your own filtering — the metadata lets you slice by quality, language, source domain. **SlimPajama** (627B tokens). Deduplicated subset of RedPajama V1. The right "small but clean" web corpus when your training budget is a single 8× H100 box and you don't have time to do your own dedup at scale. Domain-specific corpora that pair well: - **The Stack v2** — code, ~67T tokens, opt-out respected - **PG-19** — long-form prose, classic literature - **Wikipedia dumps** — factual grounding, multilingual - **arXiv** — scientific writing - **OpenWebMath** — mathematical reasoning - **SmolLM-Corpus** — Hugging Face's curated small-model training set with deduplication done **Deduplication is mandatory.** Duplicated examples poison training dynamics — the model overfits to whatever appeared multiple times. Use `datatrove` (Hugging Face) or `text-dedup` for scale; MinHash + LSH at large scale; exact-match for small corpora. ## Should I train my own tokenizer? Usually no. Adopt an existing tokenizer (Llama 3, Qwen, Mistral) and inherit a battle-tested vocabulary. You'll save days of iteration and avoid subtle bugs. Train your own only when you have a heavy domain skew — code-heavy, non-English, scientific notation. The test: measure your dataset's compression rate (bytes-per-token) on your data vs the Llama tokenizer. If you save >15%, you'll feel that gain forever in training cost *and* inference cost. If you save 3%, don't bother. Default tools: **SentencePiece BPE** (byte-level), vocab size 32K for small models, 128K+ for larger or more multilingual models. `tokenizers` from Hugging Face is the production-grade implementation. ## What training infrastructure do I need? Match to your scale: - **Single H100**: nanochat-scale only. Fine for <500M models, <50B tokens. - **8× H100 single node**: the solo-dev sweet spot. nanochat, TinyLlama, MicroLlama all train here. PyTorch FSDP2 is the right default. - **Multi-node**: when you outgrow 8× H100. DeepSpeed ZeRO-3, FSDP2 + tensor parallelism, or **torchtitan** (the recommended scale-out path in 2026 — native PyTorch, async checkpointing, scales to thousands of GPUs). - **Megatron-LM**: NVIDIA's reference for huge scale; rigid. Only use at >100 GPUs. Where to rent: Lambda Labs, RunPod, Vast.ai for spot 8× H100 at ~$15–20/hr. Modal and Beam are good for "I'd rather not manage infra at all" workflows but cost more. AWS / GCP / Azure are the most expensive on per-GPU-hour but easiest to fold into existing enterprise procurement. ## How does the post-training step work? Pre-training produces a *base model* — it completes text but doesn't follow instructions. To get a usable chat / assistant model, you post-train in three stages: **1. Supervised fine-tuning (SFT).** 10K–50K instruction examples in ChatML format. This teaches the model to respond to instructions rather than continue text. A single epoch is often enough. **2. Preference tuning.** 5K–20K preference pairs (good vs bad response to the same prompt). Use DPO, ORPO, KTO, or SimPO. ORPO merges SFT and DPO into one stage and is the simplest pipeline. The preference data is the bottleneck — generating it is harder than the training. **3. (Optional) Reinforcement learning with verifiable rewards.** GRPO or DAPO. Only for reasoning tasks where you can mechanically check the answer (math, code that compiles and passes tests). This is what powered DeepSeek R1's reasoning step-change. For most small-model projects, skip — the dataset cost is much higher than the gain on non-reasoning tasks. For a solo dev: SFT + ORPO is the right floor. Detailed framework coverage is in our [fine-tuning guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). ## Walk me through a $50 pre-training run This is roughly what happens when you run `speedrun.sh` on nanochat: 1. **Provision** an 8× H100 spot instance on Lambda or RunPod (~$15/hr). 2. **Train a tokenizer** on a slice of FineWeb (~10 minutes). 3. **Pre-train** a ~700M-parameter model on 40B FineWeb tokens (~1 hour). 4. **Mid-train** on a curated higher-quality mix (~15 minutes). 5. **SFT** on instruction data (~10 minutes). 6. **Optional RL** on GSM8K math (~15 minutes). 7. **Eval** against a standard benchmark suite (~5 minutes). 8. **Serve** via a web chat UI nanochat ships (~immediate). Total wall clock: ~3 hours. Total cost: ~$48 in compute (you'll also burn ~$5 on storage and ingress/egress). The output is a GPT-2-class model — not useful in production, but you understand the entire stack and you have a complete deployable artifact. **This is the right introduction to pre-training in 2026.** ## When does the math actually work out? The decision tree for "should I pre-train": 1. Do you have >10B clean domain-specific tokens that no open model has seen? *If no, fine-tune.* 2. Do you have $20K+ in compute budget for the training run? *If no, fine-tune.* 3. Have you exhausted fine-tuning and RAG on your problem? *If no, go back and try harder.* 4. Is the inference cost or sovereignty constraint dominant? *If yes, proceed.* 5. If you're doing this to learn — run nanochat for $50. Don't conflate "learning project" with "production project." The most common mistake is treating pre-training as a quality lever — "if I train my own, it'll be better than Llama." It will not be. The best open-weight models cost $50M+ to train and were built by teams of 30+ researchers over a year. Your $30K pre-training run will produce a model worse than the freely-available Llama 4 Scout, unless you have data Meta didn't. ## FAQ ### What's the cheapest way to actually pre-train a model end-to-end? ~$50 on an 8× H100 spot instance, using [karpathy/nanochat](https://github.com/karpathy/nanochat)'s `speedrun.sh`. You get a deployable GPT-2-quality model and complete understanding of the stack in about three hours. ### What's the difference between pre-training and fine-tuning? Pre-training builds the model from random initialization on huge web-scale data. Fine-tuning starts from someone else's pre-trained weights and adjusts them for a narrow task. Pre-training is the $10K–$1M operation; fine-tuning is the $1–$50 operation. For 99% of teams, fine-tuning is the right answer. ### How many tokens do I need to pre-train per billion parameters? Chinchilla compute-optimal is ~20:1 tokens per parameter. The 2026 inference-optimal practice for production small models is 100:1 to 1000:1 — Llama 3 used ~200:1, Liquid LFM2.5-350M used 80,000:1. For learning projects, 20:1 is enough. For production small models, push way past. ### What datasets should I use for pre-training? FineWeb (general web), FineWeb-Edu (educational filter for knowledge density), SlimPajama (cleaned, smaller scale), The Stack v2 (code), PG-19 (long-form prose). Mix domain-specifically. Deduplicate with `datatrove`. For tiny models, SmolLM-Corpus is pre-cleaned and ready. ### What's the modern small-LLM architecture I should copy? Decoder-only transformer + RMSNorm pre-norm + RoPE positional embeddings + SwiGLU FFN + grouped-query attention + no biases + tied embeddings for small models. Every 2026 open-weight model converged here. Don't innovate on architecture — innovate on data and training dynamics. ### Should I train my own tokenizer? Only if your domain compression rate is >15% better than the Llama tokenizer on your data. Otherwise adopt Llama 3, Qwen, or Mistral's tokenizer and inherit the proven vocabulary. ### What hardware do I need? Single H100 for tiny experiments. 8× H100 single node for serious solo work — this is the sweet spot. Multi-node only above that. Mac Studio Ultra is great for inference but ~10× slower than H100 for training; don't pre-train on a Mac unless it's a learning exercise. See our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) for the inference side. ### What does it cost to train a Llama-3-class model? For 7B parameters at 15T tokens (Llama-3-8B scale): ~3M H100-hours, ~$1M–2M at spot pricing. Out of reach for solo devs without funding. For Chinchilla-optimal 7B at 140B tokens: ~$30K–50K at spot. For 1B at 50B tokens: ~$1.5K. For "GPT-2 quality" at 700M / 40B tokens: ~$50. ### Why bother pre-training at all if I can just fine-tune? You shouldn't, unless you have genuinely novel domain data, a data-sovereignty constraint, an edge-deployment niche that justifies a custom tiny model, or you're learning. For everything else, [fine-tune an existing open-weight model](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). ### How long does pre-training take? nanochat ~3 hours on 8× H100. TinyLlama 1.1B (3T tokens) ~90 days on 16× A100 historically; closer to 30–45 days on modern 8× H100. Llama-3-8B-class (15T tokens) ~6+ months on a multi-node cluster. The runtime is roughly linear in tokens-per-parameter times parameter-count. ### What's the right post-training stack in 2026? SFT (10K–50K instruction examples, ChatML) → ORPO or DPO (5K–20K preference pairs) → optional GRPO/DAPO for verifiable-reward reasoning tasks. ORPO merges SFT and DPO into one stage, simplest pipeline. The framework details (TRL, Axolotl, Unsloth, MLX-LM) are covered in our [fine-tuning guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). ## Related guides - [Fine-tuning LLMs](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) — the practical alternative when pre-training is overkill - [Apple Silicon LLMs](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) — running and fine-tuning models on Mac - [Self-hosting LLMs](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) — serving infrastructure once you have a model - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) — the open-weight family you'd otherwise fine-tune - [Llama 4 guide](https://codersera.com/blog/llama-4-complete-guide-2026/) - [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) - [Qwen 3.5 guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) =============================================================== [17/20] Android Emulators in 2026: The Complete Guide for Developers URL: https://codersera.com/blog/android-emulators-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== *Last updated: May 1, 2026* The Android emulator landscape in 2026 is wider and weirder than most people realise. A QA engineer at a fintech, a casual gamer trying to run a gacha title, an indie React Native developer on an M3 MacBook Air, and a Linux power user on Fedora 41 will all reach for very different tools — and most of the "best Android emulator" lists online happily ignore that those use cases barely overlap. This guide is the version we wished existed: a pragmatic, opinionated tour of every emulator class that actually matters in 2026, with real hardware requirements, real pricing, and real trade-offs. ## TL;DR - **Building or testing an Android app:** Android Studio AVD locally for daily work; Firebase Test Lab, AWS Device Farm, BrowserStack, or Sauce Labs for matrix coverage on real devices. - **Playing Android games on a PC:** LDPlayer 9 or BlueStacks 5 on Windows; MuMu Player on a Mac; GameLoop only for Tencent titles that demand it. - **Apple Silicon Mac (M1–M4):** Android Studio's native ARM64 emulator is excellent. BlueStacks Air also runs natively. Avoid Intel-only x86 emulators that need Rosetta. - **Linux desktop:** Waydroid (container-based, near-native) for personal use; Android Studio AVD for development. - **Low-end PC, no virtualization:** See our deep dive on [Android emulators without VT](https://codersera.com/blog/10-best-android-emulators-for-pc-without-virtualization-technology-vt/) and the [best emulator for low-end PCs in 2026](https://codersera.com/blog/best-android-emulator-for-low-end-pc-2026/). - **No install at all:** Browser-based and cloud emulators — see the [browser-free emulators guide](https://codersera.com/blog/android-emulator-online-browser-free/) and the [cloud phone emulators deep dive](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). ## What an Android emulator actually is (and isn't) An Android emulator runs an Android OS image inside a virtual machine on your computer. Strictly, the term "emulator" is a small misnomer — Google's AVD, Genymotion, BlueStacks, MEmu, LDPlayer and friends all use hardware virtualization (Intel VT-x, AMD-V, or Apple's Hypervisor.framework) to execute Android natively rather than interpreting it instruction-by-instruction. That is why a 2026-vintage emulator can hit 60 FPS in a 3D game on midrange hardware, and why a 10-year-old VirtualBox-based Android image cannot. The other thing emulators are not: they are not the same as *simulators*. iOS simulators (the kind Xcode ships) run an iOS-flavoured binary on your Mac's host CPU and rely on framework shims. Android emulators boot a real Android system image — kernel, system_server, ART, the lot — which is why bug repros from an emulator are usually trustworthy. ## Who actually uses Android emulators, and why their needs differ Lumping these audiences together is how you end up reading "BlueStacks is the best emulator" advice when you're trying to debug a memory leak in a React Native app. They are not the same job. - **App developers** need the Google Android Emulator (AVD), Play Store images, fast hot-reload, debuggable system images, and the ability to spin up arbitrary API levels (Android 6 through 16). Latency and accuracy matter; FPS does not. - **QA and automation engineers** need parallel device matrices, headless runs in CI, fingerprint diversity (manufacturer, locale, screen DPI), and access to real devices for the long tail of OEM bugs. They live in Firebase Test Lab, AWS Device Farm, BrowserStack, and Sauce Labs. - **Mobile gamers** need raw FPS, low input lag, mouse-and-keyboard mapping, multi-instance, and macro support. They reach for LDPlayer, BlueStacks, MEmu, MuMu, and GameLoop. - **Casual users** running messaging apps or banking apps on a desktop want a "just works" install, low resource footprint, and ideally no Google account weirdness. Browser-based and cloud emulators dominate this segment. - **Security researchers and reverse engineers** need rooted images, x86_64 builds with full debugging surface, and frequently use Genymotion or custom AOSP builds. If a recommendation doesn't name your role, it's probably wrong for you. ## The full taxonomy of Android emulators in 2026 ### Official: Android Studio AVD Google's Android Virtual Device manager, bundled with Android Studio, is the only emulator that ships system images blessed by the Android team. In 2026 it supports API 23 (Android 6) through API 36 (Android 16), with native ARM64 system images for Apple Silicon and Linux ARM hosts. It uses Intel HAXM (deprecated, EOL 2026), Windows Hypervisor Platform (WHPX), AEHD on Windows, KVM on Linux, and Hypervisor.framework on macOS. Free. ### Consumer gaming emulators BlueStacks (now in versions 5 and 10/X), LDPlayer 9, NoxPlayer, MEmu Play, GameLoop (Tencent), and MuMu Player 12 (NetEase) all target gamers. They diverge mostly on: which Android version they ship, how aggressively they monetise via in-app ads, how clean their installer is, and how well their key-mapping editor works. ### Developer-grade alternatives Genymotion Desktop and Genymotion SaaS sit between AVD and the gaming emulators — fast to boot, scriptable via Java/Python APIs, with sensor injection (GPS, battery, network throttling). Used heavily for automated UI testing pre-Firebase Test Lab. ### Linux container approach Waydroid runs Android 13 inside an LXC container on a Wayland Linux host, with direct hardware access via the binder interface. No virtualization overhead — the Android userland is just another set of Linux processes. Free and open source. ### Browser and cloud emulators ApkOnline, appetize.io, and various cloud phone services stream a remote Android instance to a browser. No install, no VT requirement, but you are sending input and receiving video over the internet. Covered in detail in our [ApkOnline guide](https://codersera.com/blog/apkonline-android-emulator-a-comprehensive-guide/), the [free browser emulators guide](https://codersera.com/blog/android-emulator-online-browser-free/), and the [cloud phone emulators deep dive](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). For the broader category — Chrome-based options included — see [Android emulators for Chrome](https://codersera.com/blog/best-android-emulators-for-chrome-run-mobile-apps-in-your-browser/). ### Cross-OS emulators (iOS-on-Android, Android-on-iOS) The "iPhone emulator for Windows" and "iOS emulator for Mac" categories are mostly streaming services or Xcode itself. Real iOS emulation on non-Apple hardware is largely a myth — see our explainers on the [best iPhone emulator for Windows PC](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/) and the [best iOS emulator for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/). For a side-by-side of the wider mobile-emulator universe, the [32 mobile emulators review](https://codersera.com/blog/32-best-mobile-phone-emulators-reviewed-in-2025-android-ios/) is the long index. ## Consumer emulator comparison | Emulator | Vendor | Latest Android version | Min RAM | Min disk | VT required | Mac (Apple Silicon) | Price | | --- | --- | --- | --- | --- | --- | --- | --- | | BlueStacks 5 | now.gg | Android 11 (Pie 32-bit also) | 4 GB | 5 GB | Recommended | BlueStacks Air (native) | Free; Premium ad-free $4/mo | | BlueStacks X / 10 | now.gg | Cloud-streamed | 2 GB (thin client) | n/a | No | Browser | Free with ads | | LDPlayer 9 | XUANZHI | Android 9 / 11 | 2 GB (4 GB rec.) | 36 GB recommended | Yes | No | Free (ad-supported) | | NoxPlayer | Bignox | Android 9 | 2 GB | 3 GB | Recommended | Intel Macs only | Free | | MEmu Play | Microvirt | Android 9 (multi-kernel) | 2 GB | 2 GB | Recommended | No | Free | | GameLoop | Tencent | Android 7 | 3 GB | 1.5 GB | Yes | No | Free | | MuMu Player 12 | NetEase | Android 12 (x86_64) | 4 GB | 5 GB | Yes (i5-7500+) | MuMu Pro for Mac (native ARM) | Free | Two things worth calling out. First, the "free" label hides a wide range of monetisation. BlueStacks runs in-emulator ads in the launcher unless you pay for Premium; LDPlayer's installer recommends partner apps; GameLoop is mostly clean but only because Tencent makes its money once you're inside Honor of Kings or PUBG Mobile. Second, BlueStacks 10 / X is not "newer" than BlueStacks 5 in the way a normal version number suggests — it is a separate cloud-streaming product, and BlueStacks 5 remains the locally-installed flagship. ## Developer and QA emulator comparison | Tool | Type | Best for | Pricing (2026) | Real devices | | --- | --- | --- | --- | --- | | Android Studio AVD | Local emulator | Daily dev, debugger attach, hot reload | Free | No | | Genymotion Desktop | Local emulator (VirtualBox) | Sensor injection, sandboxed test runs | Free for personal; Indie €136/yr; Business €412/yr | No | | Genymotion SaaS | Cloud emulator | CI matrix, scripted teardown | $0.06/minute pay-as-you-go | No | | Firebase Test Lab | Google device farm | Robo tests, Espresso, low-cost CI | Free 10 virtual / 5 physical per day; $1/hr virtual, $5/hr physical | Yes | | AWS Device Farm | AWS device farm | Heavy parallel test runs | $0.17/device-minute or $250/month per slot (unmetered) | Yes | | BrowserStack App Live / Automate | Real device cloud | Manual + automated, large device library | Live from $29/mo; Automate from $129/mo per parallel | 2,500+ Android | | Sauce Labs | Emulator + real device cloud | Enterprise scale, secure tunnels | Virtual Cloud $149/mo; Real Device $199/mo | 7,500+ devices | The right answer for most teams is: **AVD locally + Firebase Test Lab in CI** until the Firebase device variety stops being enough, then add BrowserStack or Sauce Labs. AWS Device Farm is competitive when your stack is already on AWS or when unmetered slots beat per-minute economics. ## Hardware: what your machine actually needs The dominant variable in 2026 is whether your CPU exposes Second Level Address Translation (SLAT) and whether your OS lets the emulator use it. Anything Intel Core i3 (Haswell, 2013+) or AMD Ryzen will have SLAT; the trouble is usually that VT-x or AMD-V is disabled in the BIOS, or that Windows 11 24H2's expanded Virtualization-Based Security has claimed the hypervisor and starved your emulator of it. One important deprecation: **Google's Android Emulator Hypervisor Driver (AEHD) is being sunset on December 31, 2026**. Windows users on AVD need to migrate to the Windows Hypervisor Platform before then. Microsoft also killed Windows Subsystem for Android in March 2025, so WSA is no longer an option for running Android apps on Windows. | Use case | Min CPU | Min RAM | Recommended RAM | Disk | Notes | | --- | --- | --- | --- | --- | --- | | Single AVD (Android Studio) | i3 8th gen / Ryzen 3 / Apple M1 | 8 GB | 16 GB | ~6 GB per AVD + 8 GB SDK | SSD strongly recommended | | Multi-instance gaming (BlueStacks/LDPlayer) | i5 / Ryzen 5 | 8 GB | 16 GB | 10–40 GB | Dedicated GPU helps but not required | | Heavy QA matrix locally | i7 / Ryzen 7 / M2 Pro | 16 GB | 32 GB | 50+ GB | NVMe SSD essential | | Low-end PC / no VT | any 64-bit dual-core | 4 GB | 8 GB | 5 GB | Cloud or no-VT options only | | Apple Silicon (M1–M4) | any M-series | 8 GB | 16 GB | 15 GB | Use ARM64 system images | ## Mac vs Windows vs Linux vs Chromebook **Windows** is still the broadest target. Every consumer emulator ships a Windows installer first; AVD, Genymotion, BlueStacks, LDPlayer, MEmu, NoxPlayer, MuMu, and GameLoop all run there. The catch is the Hyper-V tug-of-war: WSL2, Docker Desktop, Windows Sandbox, and any "Memory Integrity" toggle all reserve the hypervisor, and many gaming emulators refuse to launch when Hyper-V is on. BlueStacks specifically ships a Hyper-V build to work around this. **macOS on Apple Silicon** is, somewhat surprisingly, an excellent Android emulation target — but only if the emulator ships a native ARM64 build. Android Studio AVD, MuMu Pro for Mac, BlueStacks Air, and Genymotion all qualify. NoxPlayer, MEmu, LDPlayer, and GameLoop do not have native Apple Silicon builds and either run only on Intel Macs or rely on x86 translation that punishes performance. Allocate at least 4 cores and 6–8 GB of RAM to the VM for smooth UI animations. **Linux** is the home territory of two very different choices. Android Studio AVD on KVM is fast, scriptable, and the standard for development. Waydroid is a different model entirely: it boots Android 13 inside an LXC container, sharing the host kernel, with effectively zero virtualization overhead. Waydroid is a great daily-driver for casual Android-app use on a Linux desktop and works on x86_64 and ARM64 hosts. **ChromeOS** already runs Android apps natively via the bundled Play Store runtime (ARC), which is great for users but offers no developer surface. To do real development on a Chromebook, enable Linux (Crostini) and run Android Studio AVD or Waydroid inside it; or use a cloud emulator service. See [Android emulators for Chrome](https://codersera.com/blog/best-android-emulators-for-chrome-run-mobile-apps-in-your-browser/) for the browser-only routes. ## Security and privacy: the parts vendors don't volunteer Three honest observations. **Antivirus false positives are constant.** Emulators install kernel drivers, modify hypervisor state, and inject input — behaviour that looks identical to malware to most heuristic engines. Defender flagging BlueStacks or LDPlayer is not, by itself, a sign of compromise. The actual rule: only download from the vendor's official domain, and verify the installer signature. **Bundled software is the real risk.** Several consumer emulators have shipped third-party adware or partner-app installers in the past. NoxPlayer notably had a 2021 supply-chain compromise (the BigNox update server was breached and used to ship a malicious update). BlueStacks, LDPlayer, and MEmu have all shipped opt-out installers for partner products at various times. Read the installer screens; uncheck the recommended apps. **Don't sign in with your real Google account.** Most consumer emulators require a Google account to use the Play Store. Use a throwaway. Don't connect financial apps to a desktop emulator for any reason — many banking apps refuse to run on emulators, and the ones that don't refuse do not consider the desktop a trusted device. Developer-grade tools (AVD, Genymotion, Firebase Test Lab, AWS Device Farm) have a substantially cleaner posture: no in-emulator ads, no bundled partner apps, signed installers, and clear data-handling agreements. ## The mobile QA testing emulator stack If you ship an Android app, the question isn't "which emulator" — it's "which combination of emulator and real-device cloud." 1. **Local development:** Android Studio AVD, one or two API levels (current and current-minus-2), Play Store image when you need Google services. 2. **Pre-merge CI:** Firebase Test Lab Robo crawler against virtual devices ($1/hr) catches the obvious crashes cheaply. Espresso + UI Automator suites run here too. 3. **Release candidate matrix:** Firebase Test Lab physical devices ($5/hr) or AWS Device Farm slots for the OEM-specific bugs (Samsung One UI, Xiaomi MIUI / HyperOS, Oppo ColorOS) that virtual devices cannot reproduce. 4. **Manual exploratory:** BrowserStack App Live or Sauce Labs Real Device Cloud to drive a specific Galaxy S24 / Pixel 9 / Redmi Note 13 by hand when a customer report mentions one. 5. **Performance and battery:** Real devices, full stop. Emulators do not faithfully model thermal throttling, modem behaviour, or battery curves. ## Known issues and what to avoid - **Windows 11 24H2 + emulator crashes.** Memory Integrity / VBS reserves the hypervisor; LDPlayer and MEmu often fail to launch. Either disable Memory Integrity (security trade-off) or move to a Hyper-V-aware emulator (BlueStacks, AVD with WHPX). - **Intel HAXM is gone.** If a tutorial tells you to install HAXM, the tutorial is out of date. Use WHPX on Windows, KVM on Linux, Hypervisor.framework on macOS. - **NoxPlayer supply chain history.** The BigNox 2021 incident is a cautionary tale; if you must use Nox, verify hashes and keep it isolated from work machines. - **x86 emulators on Apple Silicon.** Performance is poor and several emulators flat-out crash. Use ARM-native builds. - **"Free iOS emulator for PC" downloads.** Almost universally junkware. Real iOS development happens on a Mac with Xcode; everything else is streaming or fake. - **WSA is gone.** Windows Subsystem for Android was retired by Microsoft on March 5, 2025. Don't plan around it. - **Banking and DRM apps.** SafetyNet and Play Integrity API will flag emulators. Don't waste days trying to bypass — it's a moving target and against most apps' ToS. ## FAQ ### Is Android Studio's emulator good enough for daily development? Yes. With native ARM64 system images on Apple Silicon and WHPX/KVM acceleration on Windows and Linux, AVD boots in under 15 seconds on modern hardware and supports every API level you'll need. The friction is mostly memory: 16 GB host RAM is the comfortable floor. ### Which Android emulator is best for low-end PCs? LDPlayer 9 Lite, MEmu Lite, and NoxPlayer's lightweight profile run on 4 GB RAM machines with VT enabled. Without VT, you're looking at cloud or browser-based options. Our [low-end PC emulator guide](https://codersera.com/blog/best-android-emulator-for-low-end-pc-2026/) goes deeper. ### Can I run an Android emulator without VT-x / virtualization? Locally, performance will be unusable for most apps; the few options that work are limited. Cloud emulators bypass the requirement entirely. See [our list of no-VT emulators](https://codersera.com/blog/10-best-android-emulators-for-pc-without-virtualization-technology-vt/). ### What's the best Android emulator for a Mac with Apple Silicon? For development, Android Studio AVD with an ARM64 system image. For gaming, BlueStacks Air (native Apple Silicon build) or MuMu Pro for Mac. Avoid NoxPlayer, LDPlayer, and MEmu — they have no native ARM build. ### Is BlueStacks safe? The BlueStacks installer from bluestacks.com is signed, malware-free, and used by tens of millions of people. The risks are: antivirus false positives, in-emulator ads, and whatever Android apps you install inside it. Don't use it for banking, and use a throwaway Google account. ### Is NoxPlayer safe in 2026? NoxPlayer's 2021 supply-chain breach (NightScout) means it sits lower on the trust ladder than its competitors. If you use it, isolate it, never sign in with a real Google account, and prefer alternatives for sensitive workloads. ### What happened to Windows Subsystem for Android? Microsoft retired WSA on March 5, 2025. App support ended; there is no successor on Windows 11. For Android-on-Windows now, use BlueStacks, LDPlayer, or AVD. ### How much does Firebase Test Lab cost? Free tier: 10 virtual-device tests and 5 physical-device tests per day. Beyond that: $1 per virtual-device hour, $5 per physical-device hour, billed per minute. ### How much does AWS Device Farm cost? $0.17 per device-minute pay-as-you-go, or $250 per month per device slot for unmetered. The first 1,000 device-minutes are free. ### BrowserStack vs Sauce Labs — which should we pick? BrowserStack publishes lower entry pricing (App Live from $29/mo, Automate from $129/mo per parallel) and has the larger Android real-device library. Sauce Labs is more competitive at enterprise scale (100+ parallels) and includes 1,700+ emulators alongside its real-device cloud. For most teams under 20 parallels, BrowserStack wins on price transparency. ### Can I use Genymotion for free? Genymotion Desktop has a free tier for personal use. Commercial use requires Indie (€136/yr), Business (€412/yr), or Enterprise. Genymotion SaaS is pay-as-you-go at $0.06 per device-minute. ### Is Waydroid a real alternative to AVD on Linux? For running Android apps on a Linux desktop — yes. For Android development with a debugger — no, AVD remains the right tool. Waydroid is great when you want one or two Android apps living alongside your Linux apps with near-native performance. ### Can I emulate iOS on Windows or Android? Not really. Tools that claim to are usually skinned Android emulators with iOS-themed launchers, or they are remote streaming services. Real iOS testing requires Xcode on a Mac, or a service like BrowserStack streaming a real iPhone. See [iPhone emulators for Windows](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/) and [iOS emulators for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/). ### What about Chromebooks? Chromebooks already run Android apps natively via ARC. For development, enable Crostini Linux and run Android Studio AVD inside it, or use a cloud emulator. See [Android emulators for Chrome](https://codersera.com/blog/best-android-emulators-for-chrome-run-mobile-apps-in-your-browser/). ### Are browser-based emulators good enough for testing? For sanity-checking a build or a quick demo, yes. For automated testing against real OS behaviours, no — latency, audio, and sensor support are all limited. Background reading: [free browser emulators](https://codersera.com/blog/android-emulator-online-browser-free/), [ApkOnline guide](https://codersera.com/blog/apkonline-android-emulator-a-comprehensive-guide/), and [cloud phone emulators](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). ### What's the difference between an emulator and a simulator? An emulator boots a real Android OS image inside a VM and runs unmodified Android binaries. A simulator (the iOS kind) runs a re-compiled-for-host version of the OS and uses framework shims. Android emulators are closer to "real" devices than iOS simulators are, which is why Android engineers trust emulator bug repros more than iOS engineers trust simulator ones. ## Next steps If you're building or testing an Android app and want senior people who already know the AVD-vs-Firebase-vs-BrowserStack trade-offs cold — and who can ship without three weeks of ramp-up — we can help. [Hire a Codersera-vetted React Native or Android developer](https://codersera.com/blog/hire/hire-react-native-developer). We pre-vet for technical fit, remote-readiness, and the kind of platform judgement this guide is full of, so you spend less time interviewing and more time shipping. For the wider mobile-emulator universe, our [review of 32 mobile phone emulators](https://codersera.com/blog/32-best-mobile-phone-emulators-reviewed-in-2025-android-ios/) is the best long-form index to keep bookmarked alongside this guide. =============================================================== [18/20] iOS Simulators in 2026: The Complete Guide for Developers URL: https://codersera.com/blog/ios-simulators-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== If you build for the App Store, you live inside the iOS Simulator. It is the single most-used tool in the Apple developer toolchain, and it is also one of the most misunderstood. People reach for an "iOS emulator for Windows" the way they reach for an Android emulator, and almost everything they find is either an Android skin, a streaming service, or outright malware. There is exactly one place a real iOS Simulator runs: macOS. Everything else is a workaround. This guide covers what the Xcode Simulator is, your options on Windows and Linux (and which ones are scams), the legitimate cloud services that fill the gap, and the day-to-day Simulator features, automation, and CI patterns that real iOS teams use in 2026. *Last updated: May 1, 2026.* ## TL;DR - The Xcode Simulator is a **simulator**, not an emulator. It runs your app as a native macOS process against iOS frameworks. It does not virtualise iPhone hardware, so it is fast, and it does not boot a real iOS image, so it cannot run device-only binaries. - There is no real iOS Simulator on Windows or Linux. Anything advertised as one is either an online streaming wrapper around a real Mac, an Android launcher reskinned to look like iOS, or unsafe. - Legitimate non-Mac options fall into two buckets: **browser-streamed simulators** (Appetize.io, Smartface) and **real-device clouds** (BrowserStack App Live, Sauce Labs, AWS Device Farm, Firebase Test Lab, Kobiton, LambdaTest, Corellium). - For real teams, the workflow that scales is: develop against the local Simulator, automate with `xcrun simctl` and fastlane, run CI on macOS GitHub Actions runners with Xcode 26, and reach for a real-device cloud for release-blocking smoke tests. - If you need actual iOS engineering capacity, [hire a Codersera-vetted iOS or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer) rather than fighting the toolchain alone. ## 1. What the iOS Simulator Actually Is When you boot an iPhone in Xcode, you are not running a virtual machine. The Simulator is a regular macOS application called `Simulator.app` that loads a build of iOS frameworks compiled for the host CPU. Your app, when built for the Simulator, is also compiled for the host CPU (x86_64 on Intel Macs, arm64 on Apple Silicon). The two link together and run as a normal Mach-O process. That is why iOS builds have an `iphonesimulator` SDK distinct from `iphoneos`: the *same source code* produces different binaries depending on which CPU you target. There is no instruction translation, no hypervisor, no Rosetta-style layer. It is fast precisely because it isn't pretending to be iPhone hardware. This has consequences: - **Performance is unrealistic.** The Simulator inherits your Mac's CPU, GPU, RAM, and SSD. A scrolling animation that runs at 120fps on an M4 may stutter on an iPhone 12. - **Hardware features are simulated, not present.** No real camera frames, no real GPS, no real cellular radio, no real Secure Enclave, no real Bluetooth peripherals. - **Some libraries don't exist.** Pre-compiled vendor SDKs that ship only device slices won't link; that is why most XCFrameworks publish both slices. - **Some bugs only happen on device.** Memory pressure, thermal throttling, code-signing entitlements, push tokens, and StoreKit live transactions are device-only territory. The Simulator is for the 95% of UI, layout, navigation, and logic work where the host's speed is a feature. ## 2. Simulator vs. Emulator: Why Android Does It Differently The Android Emulator runs an actual ARM/x86 Android image inside QEMU on top of Hypervisor.framework, HAXM/WHPX, or KVM. It boots a real Android kernel. Your APK is the same APK that runs on a Pixel — full-fat *emulation* with hardware virtualisation: high fidelity, heavy on the host. The iOS Simulator does the opposite trade. It throws away fidelity for speed. You build for `iphonesimulator` and a macOS process appears — no kernel, no boot, no virtualisation. If you want a side-by-side primer on the Android side, see our [Android emulators complete guide for 2026](https://codersera.com/blog/android-emulators-complete-guide-2026/). The short version: an Android emulator is a virtual phone; the iOS Simulator is your app pretending to be on a phone. ## 3. Installing and Using the Xcode Simulator The only supported way to get the iOS Simulator is to install Xcode on macOS. As of May 2026, Xcode 26.4.1 is the current release, free from the Mac App Store or the Apple Developer downloads page. It bundles the Simulator runtime for the latest iOS, plus an installer for older iOS, watchOS, tvOS, and visionOS runtimes. You can manage runtimes from *Xcode → Settings → Components*. Day-to-day Simulator features that matter: - **Device chooser.** Pick any iPhone or iPad form factor on any installed iOS runtime. Xcode 26 ships device profiles for the iPhone 17 lineup and the iPad Pro M5. - **Network Link Conditioner.** Bundled in the *Additional Tools for Xcode* download. Throttle bandwidth and inject packet loss with presets for 3G, Edge, LTE, 5G, and "Very Bad Network." It applies to the host, so it affects every Simulator. - **Location simulation.** *Features → Location* lets you set a static GPS coordinate, simulate a freeway drive, a city run, or import a GPX file for arbitrary tracks. Critical for any maps or fitness app. - **Push notification simulator.** Drag an APNs JSON payload onto the Simulator window, or run `xcrun simctl push <device> <bundle-id> payload.json`. No APNs server, no certificates required for local testing. - **Device states.** Toggle Light/Dark mode, Dynamic Type sizes, accessibility settings, low-power mode (cosmetic), and the new Liquid Glass appearance toggle Apple introduced in iOS 26. - **Scenarios.** Simulate incoming calls, memory warnings, screenshot triggering, and Handoff between devices. - **Hardware menu.** Rotate, shake, simulate Touch ID and Face ID, trigger Home and Lock buttons, and toggle Increase Contrast. - **Drag-and-drop media.** Drop photos, videos, contacts (.vcf), or app payloads on the Simulator to install them. - **Recording.** *File → Record Screen* for video, ⌘S for screenshots, both wired into `simctl io` for headless capture. ## 4. Automating the Simulator with `xcrun simctl` `simctl` is the command-line interface to the Simulator. It is the foundation under fastlane, Detox, Maestro, and most CI scripts. Useful commands: ``` # List everything xcrun simctl list devices available # Create a fresh device xcrun simctl create "QA-iPhone-17" "iPhone 17 Pro" "com.apple.CoreSimulator.SimRuntime.iOS-26-0" # Boot, install, launch xcrun simctl boot "QA-iPhone-17" xcrun simctl install booted ./build/MyApp.app xcrun simctl launch booted com.codersera.myapp # Send a push payload xcrun simctl push booted com.codersera.myapp payload.json # Set location xcrun simctl location booted set 37.3349,-122.0090 # Open a deep link xcrun simctl openurl booted "myapp://orders/42" # Capture xcrun simctl io booted screenshot out.png xcrun simctl io booted recordVideo --codec=h264 out.mp4 # Reset xcrun simctl shutdown all xcrun simctl erase all ``` Anything you can do in the Simulator UI, `simctl` can script. CI pipelines should treat Simulators as ephemeral: create, boot, run tests, shut down, erase. Never share a Simulator between test runs — stale UserDefaults and keychain state are the most common source of flaky iOS tests. ## 5. *fastlane snapshot* and Screenshot Automation fastlane `snapshot` (the modern action is `capture_screenshots`) drives a UI test target across every device and locale you list, producing the exact screenshots the App Store expects. For 2026 submissions, you need **1320×2868** for the iPhone 6.9-inch slot and **2064×2752** for the 13-inch iPad. fastlane 2.230.0 added support for the iOS 26 device sizes and the `TabView` identifier changes Apple shipped at WWDC 25; older fastlane versions silently fail on iOS 26 simulators. The pattern looks like: ``` # fastlane/Snapfile devices(["iPhone 17 Pro Max", "iPhone 17", "iPad Pro 13-inch (M5)"]) languages(["en-US", "es-ES", "ja-JP", "de-DE"]) scheme("MyAppUITests") output_directory("./screenshots") clear_previous_screenshots(true) ``` Combined with `deliver`, you push localised screenshots straight into App Store Connect. For teams shipping multi-locale apps, this is the difference between a one-hour release ritual and a one-week marketing-team bottleneck. ## 6. CI: GitHub Actions, Xcode Cloud, Bitrise iOS CI requires a macOS runner. There are no Linux shortcuts; `xcodebuild` is macOS-only. The three real options: - **GitHub Actions `macos-15` / `macos-26` runners.** Apple Silicon, Xcode 26 pre-installed. Apple Silicon GitHub runners cut iOS test wall time roughly in half compared to the older Intel `macos-12` images. Split builds with `xcodebuild build-for-testing` followed by `test-without-building` to cache the build artefact across parallel test shards. - **Xcode Cloud.** Apple's first-party CI, billed in compute hours and tied to App Store Connect. Lowest friction for pure Apple shops; least flexible for monorepos and hybrid stacks. - **Bitrise / Codemagic / CircleCI macOS.** Specialised mobile CI with dedicated Mac infrastructure. Better caching primitives than generic CI for Pods, SPM, and DerivedData; more expensive per minute. A representative GitHub Actions step: ``` jobs: test: runs-on: macos-26 steps: - uses: actions/checkout@v4 - uses: maxim-lobanov/setup-xcode@v1 with: xcode-version: "26.4" - name: Build for testing run: | xcodebuild build-for-testing \ -scheme MyApp \ -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.0' - name: Test run: | xcodebuild test-without-building \ -scheme MyApp \ -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.0' \ -resultBundlePath TestResults.xcresult ``` ## 7. iOS on Windows and Linux: The Honest Answer You cannot run a real iOS Simulator on Windows or Linux. Apple does not ship one, and the iOS frameworks are not redistributable. What does exist falls into four categories, only the last two of which are useful for development: 1. **Android skins dressed as iOS.** Apps like the historical "iEMU" or various "iOS launcher" APKs. They run on Android and theme it. They cannot run iOS apps. Some are outright malware. Avoid. 2. **Web shells with screenshots of iOS.** "iPadian" is the canonical example: a Windows app that shows iOS-shaped UI built in Adobe AIR or .NET. It does not run iPhone apps; it runs a curated catalogue of look-alike web apps. Functionally entertainment, not development. 3. **Browser-streamed Simulators on real Macs.** Appetize.io, Smartface Cloud, BrowserStack App Live: a real Mac runs the real Simulator and streams the screen to your browser. This is legitimate, useful for demos and quick checks, and what most "iOS emulator online" reviews are actually pointing at, often without saying so. 4. **Real iOS device clouds.** Hosts of physical iPhones you connect to remotely. Sauce Labs, BrowserStack, AWS Device Farm, Firebase Test Lab, Kobiton, LambdaTest, Corellium. Slower than a local Simulator, far higher fidelity than any browser stream. For deeper context on what people pick when they want any of this on Windows specifically, see our [best iPhone emulator for Windows PC](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/) roundup and the [10 best free iPhone emulators online](https://codersera.com/blog/10-best-free-iphone-emulators-online-2025-no-download-required/) shortlist. On Mac specifically, our [best iOS emulator for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/) guide covers when reaching for anything other than the bundled Xcode Simulator makes sense. ## 8. Online iOS Emulators and Browser-Based Simulators "Online iOS emulator" almost always means: a browser interface streaming an iOS Simulator (or a real iPhone) running on someone else's hardware. The economics: someone has to own a Mac, you pay them by the minute. That is the entire business model. Pricing as of May 2026: | Service | What you get | Entry price | Honest fit | | --- | --- | --- | --- | | Appetize.io | Streamed Xcode Simulator, embeddable in a webpage, Playwright automation | Free tier (100 min/month, queued); Basic from $40/month; Premium $400/month; Enterprise from $2,000/month | Demos, sales engineering, embedded interactive tutorials, automated regression with Playwright | | Smartface Cloud | Cloud iOS dev IDE on Windows/Linux; runs real iOS builds remotely | From around $99/month per developer, enterprise quoted | JavaScript/TypeScript native iOS development from a non-Mac | | BrowserStack App Live | Real iPhones (not Simulators) streamed in browser | $29/user/month for App Live | Manual QA on real hardware, last-mile pre-release smoke testing | | LambdaTest (TestMu AI) Real Device Cloud | Real iOS devices in browser plus automation hooks | From $39/month entry tier | Cost-conscious teams that want both manual and automated mobile testing | | iPadian / iEMU / "free iOS emulators" | Not iOS. Skinned Windows or Android UIs. | Free / ad-supported | Avoid. See section 9. | For a longer survey including more niche entries, our [free iPhone emulators online comprehensive guide](https://codersera.com/blog/free-iphone-emulators-online-a-comprehensive-guide/) and [best virtual mobile device emulators](https://codersera.com/blog/best-virtual-mobile-device-emulators-run-android-and-ios-apps/) piece walk through specific UI quirks and trial limits. ## 9. Known Issues and What to Avoid (the iOS-Emulator-for-Windows Scams) Search "iOS emulator Windows" on any major engine and the first page is almost entirely SEO content optimised for the keyword, not for the developer. Some patterns to watch for: - **Names that recycle dead products.** iPadian, iEMU, AIR iPhone, Cider, MobiOne. None of these are functioning iOS environments today. Several of the modern download pages bearing these names serve installers signed with throwaway certificates. - **"Free download, no Mac required."** Real iOS Simulators require iOS framework binaries that Apple does not redistribute. A free, signature-light installer for an "iOS emulator" is almost always either an empty Electron / .NET shell or a malware dropper. - **Survey walls and "human verification."** A legitimate developer tool never asks you to fill in a CPA offer to unlock a download. - **YouTube tutorials with millions of views.** The economics of monetised YouTube reward "I ran iOS on Windows!" thumbnails. Most of these videos demonstrate iPadian (a web wrapper) or an Android emulator booted into a custom launcher. - **"Cloud iOS" with a flat $5 lifetime fee.** Mac Mini hosting costs more than that per month wholesale. The math doesn't work; the service either disappears in weeks or it's running on stolen credentials. Three rules that filter 95% of the noise: 1. If it doesn't require a real Apple Developer Program account or device serials, it isn't running real iOS. 2. If it claims to run arbitrary `.ipa` files in your browser without uploading them to a real iOS host, it's lying. 3. If the company has no LinkedIn footprint, no funding history, and no enterprise customers, treat the binary as untrusted. ## 10. Real-Device Clouds: When Simulators Aren't Enough Simulators cannot certify a release. Camera capture, ARKit, CoreNFC, real cellular conditions, real APNs delivery, real StoreKit purchases, thermal throttling under load — all device-only. Real-device clouds rent you a physical iPhone over the network. | Provider | Pricing model | Indicative price | Strengths | Watch-outs | | --- | --- | --- | --- | --- | | BrowserStack App Live / App Automate | Per-user or per-parallel | App Live $29/user/month; App Automate Pro $249/month | Largest device fleet (30,000+), instant access, strong manual QA UX | Per-parallel pricing scales fast for automated suites | | Sauce Labs Real Device Cloud | Concurrency + minutes | Typically $3,000–$75,000+ /year, quote-based | Enterprise-grade, private device options, mature Appium pipelines | No published self-serve tier; sales-led | | AWS Device Farm | Pay-as-you-go or unmetered slot | $0.17/device-minute or $250/slot/month | Pure usage billing, integrates with AWS IAM, no minimums | Smaller iOS fleet; older devices over-represented | | Firebase Test Lab (Apple devices) | Per-hour | $5/hr real device, $1/hr virtual; free tier 5 real / 10 virtual tests/day | Cheapest entry, deep Google Cloud integration, good for XCTest | Limited device list, no manual exploratory mode | | Kobiton | Per-user / per-device subscription | Quote-based, mid-market positioning | Visual testing add-ons, on-prem option for regulated industries | Higher per-device cost than AWS or Firebase | | LambdaTest (TestMu AI) | Tiered subscription | From $39/month entry; HyperExecute and AI test agents at higher tiers | Cheapest premium-feeling self-serve, broad cross-browser overlap | Newer iOS device additions sometimes lag | | Corellium | Virtualised iOS on ARM hosts | Quote-based, security-research positioning | Snapshot, fuzz, jailbreak research, kernel debugging | Not for App Store-flow testing; no App Store / TestFlight on virtual devices | For most product teams the right shape is: develop on the local Simulator, run XCUITest in CI on macOS runners, and farm out a small release-blocking suite plus exploratory manual passes to a real-device cloud. For a broader lens on cloud-phone tooling that includes Android, see our [best cloud phone emulators in-depth guide](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). ## 11. Debugging Patterns That Save Days - **Reset before reproducing.** Most "works on my machine" iOS bugs are stale UserDefaults or Keychain state. `xcrun simctl erase all` first, reproduce second. - **Use the same Simulator family across the team.** A test that passes on iPhone 17 Pro and fails on iPhone SE is a real bug; a test that passes on your laptop and fails in CI because CI uses a different device is wasted hours. - **Always test the smallest supported width.** The cheapest source of layout bugs is the smallest iPhone in your support matrix; UI test that device first, not last. - **Reach for Instruments early.** Xcode 26's Processor Trace and Power Profiler instruments report per-subsystem energy and instruction-level traces; they catch animation jank and battery regressions that pure XCTest will miss. - **Snapshot tests over screenshot tests.** Pointfree's `swift-snapshot-testing`, run in the Simulator, gives diffable PNG and text snapshots in pull-request reviews. Fastlane `snapshot` is for marketing; `swift-snapshot-testing` is for engineering. - **Don't trust the Simulator for performance numbers.** Profile on the slowest device in your support matrix. Frame timing and memory pressure on a Mac M4 are not iPhone numbers. ## 12. Next Steps The Simulator is a fast feedback tool. It is not a substitute for a real iPhone, and it is not available outside macOS. Build the right pipeline: 1. Develop locally against the Xcode 26 Simulator. 2. Automate with `xcrun simctl` and fastlane. 3. Run CI on macOS Apple-Silicon runners. 4. Validate releases on a real-device cloud sized to your team. 5. Profile performance and battery on actual hardware. If the gap is people, not tools, [hire a Codersera-vetted iOS or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer). We pre-screen for Xcode and Simulator fluency, fastlane and CI experience, and shipping App Store releases under deadline pressure, so you skip the months it usually takes to find an engineer who is genuinely remote-ready and can extend your team without onboarding overhead. ## FAQ ### Is the iOS Simulator really not an emulator? Correct. It does not emulate iPhone hardware or boot a real iOS image. It loads iOS frameworks compiled for your Mac's CPU and runs your app as a native macOS process. ### Can I run the iOS Simulator on Windows or Linux? No. The Simulator depends on macOS-specific frameworks and Apple-redistributable binaries that ship only with Xcode. Anything claiming otherwise is either streaming a real Mac to your browser or isn't running iOS at all. ### What about a Hackintosh or a macOS VM on Windows? Technically possible, against Apple's licence terms (macOS is licensed only on Apple-branded hardware), and a fragile setup. Not recommended for a professional toolchain. Buy a Mac mini or use a cloud Mac like MacStadium or Scaleway Mac. ### Is Appetize.io a real iOS Simulator? Yes — it streams a real Xcode Simulator running on Apple-branded hardware in their cloud. It is not running on your local Windows machine; the pixels are coming over the wire. ### Can I submit an app to the App Store using only an online emulator? You still need an Apple Developer Program account and code-signing identities. You can do the development and submission from a streamed Mac (such as MacStadium) but not from any "iOS emulator for Windows" — those don't compile or sign iOS binaries. ### What's the difference between Simulator and a real iPhone for testing? Simulator covers UI, navigation, layout, accessibility, deep links, push payload formatting, and most logic. A real iPhone is required for camera, ARKit, CoreNFC, Bluetooth peripherals, Secure Enclave behaviour, real cellular conditions, real push delivery, and StoreKit live transactions. ### How do I test push notifications without a server? Drag an APNs JSON payload onto the Simulator window, or run `xcrun simctl push booted <bundle-id> payload.json`. The payload must include the `aps` dictionary and a `Simulator Target Bundle` key matching your app. ### How do I simulate a slow network in the Simulator? Install *Additional Tools for Xcode*, open Network Link Conditioner from System Settings, and choose a preset like 3G or "Very Bad Network." It applies to all network traffic on the host, including the Simulator. ### Is BrowserStack the same as Appetize.io? No. Appetize streams a Simulator. BrowserStack App Live streams a real iPhone. Higher fidelity, slower frame rate, more useful for release-blocking QA. ### What's the cheapest way to run iOS tests in CI? GitHub Actions `macos-latest` runners are included in the free tier for public repos and metered per-minute for private. For real-device coverage, AWS Device Farm at $0.17/device-minute or Firebase Test Lab at $5/hr are the lowest entry points. ### Can I run my app on iOS 17 if I only have Xcode 26? Yes. Older iOS Simulator runtimes are downloadable from *Xcode → Settings → Components*. You can keep iOS 17, 18, 26 runtimes side by side and target each from `xcodebuild`. ### Is Corellium an iOS emulator? Corellium virtualises iOS on ARM server hardware. It is closer to real iOS than the Xcode Simulator (it boots a real iOS kernel), but it does not have access to the Apple App Store, TestFlight, or App Store StoreKit. It is positioned for security research and reverse engineering, not for general-purpose product development. ### Does the Simulator support Apple Pay? Apple Pay is testable in a sandbox-only mode on the Simulator. Real production Apple Pay transactions require the Secure Enclave and only run on a physical device. ### Why does my app crash on a real iPhone but not in the Simulator? Common causes: differing CPU architectures revealing assumptions about pointer size or alignment; entitlements that only apply to signed builds; missing background-mode capabilities that the Simulator silently ignores; race conditions exposed only by slower device storage; or vendor SDKs whose Simulator slice is a no-op stub. ### Should I hire an iOS developer or learn this myself? If you're shipping a one-off internal tool, learn it. If iOS is a revenue surface, [hire a Codersera-vetted iOS or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer) and extend your team instead of bottlenecking on the toolchain. =============================================================== [19/20] Mobile App Testing in 2026: The Complete Guide URL: https://codersera.com/blog/mobile-app-testing-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== Mobile testing in 2026 is no longer a single discipline. It is a stack — unit tests on the JVM or a Swift target, integration tests against a mocked backend, instrumented UI tests on emulators, end-to-end flows on real devices in a cloud farm, and post-release crash and performance telemetry feeding the next sprint. Treat any one layer as the whole strategy and you will ship regressions. This guide covers how serious mobile teams structure that stack, what the 2026 tooling landscape looks like, what device clouds actually cost, and where the sharp edges are — written for engineers, QA leads, and tech leads making real build-vs-buy and in-house-vs-outsource calls, often alongside [Codersera-vetted mobile engineers](https://codersera.com/blog/hire/hire-react-native-developer) who have to live with the consequences. *Last updated: May 1, 2026.* ## TL;DR - The 70/20/10 testing pyramid (unit / integration / E2E) still holds for mobile, but device fragmentation pushes a thicker integration layer than on web. - Native frameworks — Espresso for Android, XCUITest for iOS — remain the gold standard for low-flake instrumented tests. Cross-platform work goes to Appium 2 (modular driver model), Maestro (YAML, black-box, <1% flake), or Detox (gray-box, React Native). - For Flutter, `integration_test` is the floor and Patrol is the practical ceiling — it crosses the native boundary that `integration_test` cannot. - Device clouds are not interchangeable. Firebase Test Lab is cheapest for short Android runs; AWS Device Farm wins on unmetered concurrency; BrowserStack and Sauce Labs lead on real-device breadth and enterprise features; LambdaTest competes on price; Kobiton focuses on session-based manual testing. - Crash and performance telemetry (Sentry, Firebase Crashlytics, Firebase Performance) is part of the test stack now — shift-right is how you cover the device matrix you cannot afford to test pre-release. - The right answer is almost always hybrid: emulators for unit and most UI tests in CI, a small real-device pool for nightly E2E, and a cloud farm for release-candidate matrix runs. ## What makes mobile testing structurally hard Most of the difficulty in mobile testing comes from things that simply do not exist on web. A web target is Chromium, Firefox, and WebKit on a handful of viewport sizes. A mobile target is two operating systems with multi-year version ranges still in active use, thousands of Android OEM SKUs with their own quirks, deep OS integrations (permissions, biometrics, push, deep links, background execution), and a network stack that ranges from gigabit Wi-Fi to a 3G cell on a train. Add the asynchronous nature of mobile UIs — animations, view recycling, threading models, and platform-imposed lifecycles — and "did the button do the thing" becomes non-trivial. Espresso and XCUITest exist primarily because Selenium-style polling does not work on a UI thread rendering at 120 Hz. Both ship idling resources or built-in synchronization; both still struggle with WebViews, custom Compose or SwiftUI components, and any animation that does not advertise its end state. If you are not yet comfortable with the emulator side of the equation, our [complete guide to Android emulators](https://codersera.com/blog/android-emulators-complete-guide-2026/) and the broader [survey of 32 mobile emulators](https://codersera.com/blog/32-best-mobile-phone-emulators-reviewed-in-2025-android-ios/) are useful background. ## The mobile testing pyramid in 2026 The classical Cohn pyramid — many fast unit tests, fewer integration tests, very few end-to-end tests — still maps cleanly onto mobile. The ratios most teams converge on are 60–70% unit, 15–25% integration, and 10–15% UI / E2E. Mobile tilts slightly more toward integration than web because so much of the value of a mobile app sits at the seam between your code and platform APIs (notifications, permissions, storage, background tasks). | Layer | What it covers | Where it runs | Typical tooling | Target run time | | --- | --- | --- | --- | --- | | Unit | Pure logic, view models, reducers, formatters | JVM (Android), Swift host process | JUnit, Kotest, XCTest, Quick / Nimble, Jest (RN) | < 5 ms / test | | Component / Widget | Single Compose / SwiftUI / RN / Flutter widget | JVM with Robolectric, host XCTest, Jest, flutter_test | Compose UI Test, ViewInspector, React Native Testing Library, flutter_test | < 100 ms / test | | Integration | Module + dependencies, mocked network, real DB | Emulator / simulator | AndroidX Test, XCTest, MockWebServer, Hilt / Koin test modules | 1–10 s / test | | UI / Instrumented | Single screen or short flow on device | Emulator (CI) and a few real devices | Espresso, XCUITest, Compose UI Test, EarlGrey 2 | 10–60 s / test | | E2E | Cross-screen user journeys, real backend or staging | Real devices, often via cloud | Maestro, Appium 2, Detox, Patrol | 1–5 min / flow | | Production telemetry | Crash, ANR, performance, regression detection | End-user devices | Sentry, Crashlytics, Firebase Performance | Continuous | "Shift-left" means push coverage into the bottom three rows, which run on every commit. "Shift-right" — staged rollouts, feature flags, crash telemetry — covers the device and locale matrix you cannot afford to enumerate pre-release. Both are necessary. ## Test types beyond functional Functional correctness is table stakes. The categories that distinguish a mature mobile test plan in 2026: - **Performance.** Cold start, time-to-first-frame, scroll jank, frozen frames. Android's Macrobenchmark library, Instruments on iOS, and Firebase Performance Monitoring in production cover this. - **Network conditions.** Test on simulated 3G, packet loss, and offline-then-reconnect transitions. BrowserStack, Sauce Labs, and most cloud farms expose network shaping. Locally, the Android emulator's network speed flag and Network Link Conditioner on macOS get most of the way there. - **Battery and thermal.** Background work that drains battery is a leading cause of one-star reviews. Android's Battery Historian and iOS's MetricKit are the tools of record. - **Accessibility.** TalkBack and VoiceOver flows, contrast, dynamic type, RTL. The Accessibility Scanner on Android and Accessibility Inspector on iOS catch the easy cases; manual screen-reader walkthroughs catch the rest. - **Security.** Static analysis (MobSF, Android Lint security checks), TLS pinning verification, root/jailbreak detection, OWASP MASVS coverage. - **Localization.** Long-string overflow, RTL mirroring, locale-specific date and number formats. Pseudo-localization in CI catches most truncation bugs before a translator ever sees the build. - **Beta and dogfood.** TestFlight on iOS, Play Console internal and closed testing on Android, plus Firebase App Distribution for ad-hoc cross-platform builds. ## The framework landscape The framework you pick determines what kind of flake you fight, how fast tests run, and how many engineers can read them. The 2026 shortlist: | Framework | Platforms | Approach | Language | Strengths | Tradeoffs | | --- | --- | --- | --- | --- | --- | | Espresso | Android | Gray-box, in-process | Kotlin / Java | UI-thread sync, low flake, Compose support | Android only; learning curve for IdlingResources | | XCUITest | iOS | Black-box, out-of-process | Swift | Apple-maintained, ships with Xcode, strong on simulators | iOS only; flakier on real devices than emulators | | EarlGrey 2 | iOS | White-box on top of XCUITest | Objective-C / Swift | Better synchronization than vanilla XCUITest | Small community outside Google; XCUITest is usually enough | | Appium 2 | iOS, Android, more | WebDriver, modular drivers | Any (JS, Java, Python, Ruby, C#) | Cross-platform, huge ecosystem, real and virtual devices | Slower than native; setup complexity; driver versions matter | | Maestro | iOS, Android, RN, Flutter, web | Black-box via accessibility layer | YAML | 10–15 min to first test, <1% flake, MaestroGPT for authoring | Less powerful for deep state assertions; YAML scales awkwardly past ~200 flows | | Detox | React Native (iOS, Android) | Gray-box, JS-thread aware | JavaScript / TypeScript | Idle-state synchronization, flake <2% on RN | RN-specific; 2–4 hour setup; brittle on native modules | | Flutter integration_test | Flutter (iOS, Android, web, desktop) | In-process via Flutter driver | Dart | Ships with Flutter SDK, fast, good widget control | Cannot drive native UI (system permissions, other apps) | | Patrol | Flutter (iOS, Android) | Wraps integration_test + native bridge | Dart | Drives native dialogs, permissions, Wi-Fi, biometrics | LeanCode-maintained; younger than integration_test | The practical 2026 default for a greenfield app: - **Native Android:** JUnit + MockK for unit, Compose UI Test + Espresso for instrumented, Maestro for E2E. - **Native iOS:** XCTest for unit, XCUITest for instrumented, Maestro or Appium for E2E if you need a single tool across platforms. - **React Native:** Jest + React Native Testing Library, Detox for deep RN E2E, Maestro for read-the-YAML-and-understand-it E2E. - **Flutter:** flutter_test, integration_test, Patrol for anything that crosses the native boundary. For React Native specifically, Detox's gray-box approach gives lower flake on the RN bridge but higher setup cost; Maestro's YAML brings time-to-first-test under fifteen minutes at the cost of less surgical control. Mature RN teams often run both. ## Devices: emulators, simulators, real devices, and the cloud Where a test runs is as important as how it is written. The four tiers, in increasing order of fidelity and cost: 1. **Local emulator (Android) or simulator (iOS).** Free, fast, scriptable. The iOS Simulator is genuinely close to a real device because it shares much of the underlying system; the Android Emulator with Google APIs is also strong but does not exercise OEM skin behavior. Most unit, component, and instrumented tests should run here. See our [Android emulators guide](https://codersera.com/blog/android-emulators-complete-guide-2026/) for a detailed comparison. 2. **Cloud emulator.** Same fidelity as local but parallelizable. Firebase Test Lab virtual devices, BrowserStack App Live virtuals, Genymotion Cloud. Useful for matrix runs without the local hardware bill. Our [cloud phone emulators guide](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/) goes deeper. 3. **Local real device.** A handful of "reference" devices — typically a current Pixel, a current iPhone, one mid-tier Android, and one older iPhone — wired to the workstation or to a self-hosted Bitrise / Codemagic agent. 4. **Cloud real-device farm.** Hundreds to thousands of physical devices in a data center, accessed by API or browser. Required for OEM-specific regressions, biometric flows, and any meaningful pre-release device matrix. For teams demoing without hardware, our roundups of [iPhone emulators for Windows](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/), [iOS emulators for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/), [virtual mobile device emulators](https://codersera.com/blog/best-virtual-mobile-device-emulators-run-android-and-ios-apps/), [free online iPhone emulators](https://codersera.com/blog/10-best-free-iphone-emulators-online-2025-no-download-required/), and [ApkOnline](https://codersera.com/blog/apkonline-android-emulator-a-comprehensive-guide/) separate legitimate options from snake oil. ## Device cloud comparison and pricing This is the table teams ask for and almost never find with real numbers in one place. All prices are public list pricing in May 2026 and round to the nearest sensible unit. Enterprise contracts are routinely 30–60% off list, and almost every vendor will negotiate. | Provider | Best for | Real / virtual | Pricing model | Entry price | Notes | | --- | --- | --- | --- | --- | --- | | Firebase Test Lab | Android matrix runs in CI | Both | Per device-hour, per-minute billing | $1/hr virtual, $5/hr physical (Blaze plan); free daily quotas on Spark | Cheapest for short Android runs; iOS support is limited. | | AWS Device Farm | Teams already on AWS, unmetered concurrency | Real (and remote access) | Per device-minute or unmetered slot | $0.17 / device-minute, or $250 / slot / month unmetered | Unmetered slots are the win — predictable cost at high volume. | | BrowserStack App Live | Manual, exploratory testing | Real | Per user / month | From ~$39 / user / month (annual) | Strong device breadth, geolocation, network sim. | | BrowserStack App Automate | Appium / XCUITest / Espresso CI | Real and virtual | Per parallel session | From ~$249 / month for App Automate Pro | Unlimited minutes; pay for parallels. | | Sauce Labs Real Device Cloud | Enterprise mobile + web combined | Real and virtual | Concurrency + minutes, annual | From ~$199 / month entry; enterprise commonly $20k–$75k+ / year | Real Device Access API (2026) for programmable infra. | | LambdaTest (TestMu AI) | Cost-conscious teams, web + mobile | Real and virtual | Per user / parallel | Real devices from $39 / month | Six product tracks; biometrics and camera injection included at entry. | | Kobiton | Manual + scriptless automation | Real | Minutes / month tiers | From $83 / month (500 min) to $399 / month (3000 min) | Strong on session-based manual testing and AI-assisted scripting. | | Codemagic / Bitrise | CI compute, not a device farm | Build agents | Per minute or seat | Codemagic from $0.095 / min macOS premium; Business $299 / month | Pair with Firebase Test Lab or BrowserStack for device coverage. | Two notes. First, "unlimited minutes" almost always means "limited parallels." Second, virtual-device cloud is only competitive with self-hosted CI emulators if your CI minutes are expensive (GitHub-hosted macOS) or your tests are slow to start. ## CI/CD integration The mobile CI pipeline in 2026 typically looks like this on every PR: lint and static analysis, unit tests, component tests, instrumented tests on a single emulator, build the debug APK / IPA. On merge to main: full instrumented matrix on Firebase Test Lab or BrowserStack, E2E on Maestro / Detox / Appium against a staging build, deploy to internal track and TestFlight. Nightly: full device matrix, performance benchmarks, security scans. The four CI choices most teams pick from: - **GitHub Actions.** Cheapest for Android; macOS minutes are 10× Linux minutes, which makes iOS painful at scale. Good for teams with light iOS volume. - **Bitrise.** Mobile-first, with prebuilt steps for Fastlane, code signing, Firebase Test Lab, App Store Connect. Stack stability is its main selling point. - **Codemagic.** Mobile-first, Flutter-native, pay-per-minute by default; Business plan at $299 / month gives unlimited macOS minutes for predictable spend. - **CircleCI.** Strong general-purpose CI with macOS support; better for teams that already have non-mobile workloads on it. Two rules apply regardless. Cache Gradle and Pods aggressively — half of any mobile pipeline's wall time is dependency resolution. And keep code signing off developer machines; Fastlane Match or your CI provider's managed signing is non-negotiable past three engineers. ## Crash, performance, and the shift-right side of the stack You cannot test every device-locale-OS combination pre-release. You can, however, observe what happens when real users hit the matrix. Crash and performance telemetry is now part of the test stack, not an afterthought. - **Firebase Crashlytics.** Free, deep Firebase integration, groups crashes by stack trace. Strong default for Android-led teams; iOS support is solid. - **Sentry.** Cross-platform (mobile, web, backend), per-event detail rather than aggregation, release-health metrics, performance tracing. Modern SDKs add roughly 1% CPU overhead. Better when the same team owns mobile and backend. - **Firebase Performance Monitoring.** App start time, network request latency, custom traces. Pairs with Crashlytics. - **Play Console / App Store Connect vitals.** ANRs, excessive wakeups, crash-free user rate. Free, authoritative, often the first place a regression shows up. The pattern that works: gate releases on crash-free-user-rate thresholds (typically 99.5%+ paid, 99%+ free) and tie staged rollouts to those gates. A rollout that auto-pauses on regression is worth more than another hundred E2E tests. ## Cost reality and when to outsource A realistic 2026 mobile test budget for a mid-size product team: - CI compute: $300–$2,000 / month depending on iOS volume. - Device cloud: $500–$5,000 / month for one app. - Crash and performance telemetry: $0 (Crashlytics) to $2,000 / month (Sentry at scale). - Local device lab: a few thousand dollars one-off, plus $200–$500 / month maintenance. - QA headcount: one QA engineer per three to five mobile engineers. Build-vs-buy decisions worth thinking through: - **Self-hosted lab vs cloud farm.** Below ~50 daily runs, cloud wins on TCO. Above that, a small in-house lab pays back inside a year — but only if someone owns it. - **In-house automation vs outsourced QA.** Outsource regression and exploratory testing on stable features. Keep framework ownership and CI in-house — that is where knowledge compounds. - **Generalist engineers vs specialist SDETs.** Up to ten engineers, generalists work. Past that, a dedicated mobile SDET role pays for itself. ## Known issues and sharp edges - **Compose and SwiftUI flakiness.** Both modern UI toolkits sometimes confuse the underlying test frameworks' idle detection. Animations that loop forever or use spring physics are the most common offenders. Disable animations in test builds. - **WebViews.** Espresso and XCUITest both treat WebViews as a black box. You either drop into Espresso-Web / WKWebView APIs or accept that those flows go to E2E tools like Appium. - **Permissions and system dialogs.** Anything that pops the OS-level permission sheet breaks pure-Flutter, pure-RN tools. Patrol (Flutter), Maestro, and Appium can drive those dialogs; integration_test and Detox cannot. - **Real-device flake.** Real iPhones in cloud farms are noticeably flakier than simulators because they share devices across tenants, get rebooted between sessions, and occasionally lose Wi-Fi. Plan for retries; do not gate every PR on real-device E2E. - **Native module upgrades.** A React Native or Flutter version bump frequently breaks Detox or Patrol. Pin versions and treat the bump as a project, not a chore. - **Code signing.** The most common reason a pipeline goes red is an expired profile. Automate via Fastlane Match. - **Cloud-farm queueing.** Specific models have queues at peak hours; pin to a device family, not a model. - **Test data.** Tests sharing a staging account fight each other. Provision per-test users or use seeded fixtures. ## FAQ ### What is the difference between mobile testing and web testing? Mobile testing has to deal with multiple operating systems, hundreds of OEM device variations, deep platform integrations, variable network conditions, and battery and thermal constraints. Web testing is mostly three browser engines and a handful of viewport sizes. The mobile testing pyramid therefore tends to be flatter, with relatively more integration and device-level testing. ### Should we use Espresso and XCUITest, or a cross-platform tool? For instrumented tests on a single platform, native frameworks are faster and less flaky. For flows that need to behave identically on both platforms, a cross-platform tool (Maestro, Appium 2) reduces duplication. Most mature teams use both. ### Is Appium 2 still relevant in 2026? Yes. The modular driver model decoupled the core server from platform drivers, making it lighter and easier to scale in containers. It remains the most flexible option when you need to drive iOS, Android, and other targets from a single suite in any major language. ### Maestro or Detox for React Native? Detox if you want gray-box JS-thread synchronization and your engineers will own the suite — flake under 2%, setup time 2–4 hours. Maestro if QA or product will help author flows — YAML, time-to-first-test under 15 minutes, flake under 1%. Many teams use both. ### What is the difference between integration_test and Patrol for Flutter? `integration_test` ships with Flutter and can drive widgets in the app's own tree. Patrol wraps `integration_test` and adds a native bridge so your tests can also tap system permission dialogs, toggle Wi-Fi, drive biometrics, and interact with other apps. Use Patrol whenever your test crosses the native boundary. ### How many real devices do we actually need? A defensible local matrix is one current and one previous iPhone, one current and one budget Android, plus whichever device represents your largest user segment in production. Anything beyond that should live in a cloud farm. Look at your Crashlytics or Sentry device breakdown — five devices typically cover 60–70% of your real users. ### Which device cloud is cheapest? Firebase Test Lab is cheapest for short Android runs ($1 / hr virtual, $5 / hr physical, with free daily quotas). LambdaTest is the cheapest entry point for real devices in a self-serve plan ($39 / month). AWS Device Farm wins when you need predictable cost at high volume thanks to its $250 / slot / month unmetered option. ### Can we replace E2E tests with crash analytics? No, but they cover different gaps. E2E tests catch regressions in flows you specifically wrote tests for. Crash analytics catches regressions in flows you did not anticipate, on devices and OS versions you did not test. You need both. ### How do we keep flaky tests under control? Three habits. Quarantine flaky tests in a separate suite that does not block merges, but track time-in-quarantine and treat it as tech debt. Use frameworks with built-in synchronization (Espresso, XCUITest, Maestro, Detox) instead of `sleep()` calls. Disable animations in test builds, and prefer test ID accessibility identifiers over text-based locators. ### Should we test on iOS Simulator or real iPhones? Both. The Simulator is fine for unit, component, and most XCUITest runs. Real iPhones catch issues that only show up on hardware: camera, biometrics, Bluetooth, push, thermal performance, and UIKit edge cases. Run real-device tests nightly and on release candidates. ### What about manual testing — is it dead? No. Exploratory manual testing finds bugs no automated suite will. The 2026 shift is toward making manual testing exploratory rather than scripted: anything you would write a script for, automate. ### How do we test for a region we have no devices in? Use a cloud farm with regional devices for matrix runs, and lean on Crashlytics or Sentry breakdowns by country, locale, and carrier. Simulate that region's network conditions in CI too — a flaky 3G connection in Lagos behaves nothing like Wi-Fi in Mountain View. ### What is "shift-left" vs "shift-right" in mobile testing? Shift-left moves testing earlier — unit, component, static analysis, contract — so regressions are caught at commit time. Shift-right pushes observation into production: staged rollouts, feature flags, crash and performance telemetry. Shift-left covers what you know to test; shift-right covers what you did not. ### In-house QA or outsourced QA? Outsource what is repetitive and stable: regression runs, exploratory testing of mature features, localization. Keep in-house anything that compounds knowledge: framework ownership, CI maintenance, performance benchmarking, per-feature test planning. Outsourcing the framework itself freezes it at the contractor's day-one skill level. ## Next steps Building from scratch, start at the bottom: get unit and component coverage above 60% before investing in E2E. Scaling an existing strategy, audit flake rate, CI wall time, and crash-free user rate — those three numbers tell you where the next dollar goes. Hiring for any of this, the bottleneck is rarely "knows Espresso" — it is engineers who reason about framework, CI, device strategy, and production telemetry as one system. [Hire a Codersera-vetted mobile or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer) when you need someone who has shipped this end-to-end before, not just written tests against a tutorial app. =============================================================== [20/20] Software Testing in 2026: The Complete Engineer's Guide URL: https://codersera.com/blog/software-testing-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== *Last updated: May 1, 2026.* Testing is the part of the job most engineers say they care about and quietly skip when deadlines tighten. That gap is the whole problem. In 2026, with AI agents committing code faster than humans can review, testing is a load-bearing part of the engineer's job description — not a QA team's chore. This guide is for the engineer who owns quality: writes the code, writes the tests, and is paged at 3 a.m. when both fail. We cover what is real now — the testing trophy versus the pyramid, where snapshot tests still earn their keep, the AI-augmented testing wave, the specific frameworks worth your time per ecosystem, flaky test management, performance and chaos testing, ephemeral environments, and when staffing a dedicated SDET pays off. ## TL;DR - **Forget the pyramid as dogma.** Use Kent C. Dodds' testing trophy for frontends and integration-heavy services; keep a pyramid for backends with deep business logic. The right shape depends on what your code is mostly doing — gluing systems together or computing things. - **Coverage is a vanity metric on its own.** 80% line coverage with weak assertions catches almost nothing. Pair coverage with mutation testing (Stryker, PIT, cargo-mutants) before you trust a number. - **Playwright won.** 33M weekly npm downloads, 45% adoption among QA pros in 2026, real WebKit support, free parallelization. Cypress is fine if you already run it; do not start a new project on it. - **Vitest is the default for new JS/TS projects.** 5x faster cold starts and ~28x faster watch reruns than Jest 30. Angular 21 made it the default. Stay on Jest only if migrating costs more than it earns. - **AI-generated tests are real but dangerous.** Claude Code and Cursor write plausible tests fast; they also write tests that pass without asserting anything meaningful. Mutation testing is now table stakes for AI-written suites. - **Flaky tests cost ~6–8 hours per engineer per week.** Quarantine them, do not retry them blindly. Test impact analysis (Datadog, Bazel, Nx) is the highest-ROI CI investment most teams skip. - **Hire engineers who treat testing as part of the job** before you hire a dedicated QA team. [Codersera-vetted engineers](https://codersera.com/blog/hire) ship code with tests, not behind them. ## Testing Fundamentals: Pyramid, Trophy, and What Actually Matters in 2026 The test pyramid — many unit tests, fewer integration, very few end-to-end — was Mike Cohn's 2009 model and dominated thinking for over a decade. It is still right for some workloads. It is wrong for many others. Kent C. Dodds proposed the testing trophy in 2018: static analysis at the base, then unit, then a fat middle layer of integration tests, then a thin layer of E2E. The argument is simple and increasingly true: most modern applications are integration code. A React component that renders a list, fetches data, handles errors, and writes back to a store is not a unit — it is a small system. Test it as a small system or your tests are not testing what users actually do. The 2026 reality is that both shapes are valid and the choice should be deliberate: - **Backend services with heavy domain logic** — pricing engines, scheduling, fraud scoring — earn the pyramid. Pure functions deserve isolated, fast unit tests. - **Frontends, BFFs, and orchestration services** — anything where the value is in how pieces connect — earn the trophy. Integration tests with realistic boundaries (real DOM, real DB, mocked third-party APIs) catch the bugs your users hit. - **Static analysis is non-negotiable.** TypeScript strict mode, ESLint with `typescript-eslint`, Ruff or Pyright for Python, `golangci-lint` for Go. Cheap tests that run on save eliminate entire bug classes before you write a single assertion. If your team is religiously chasing 80% unit coverage on glue code that mostly calls other functions, you are writing the wrong tests. If you have one giant Cypress suite and no isolated tests for a regex-heavy parser, same problem. ## Types of Tests and When Each One Earns Its Keep | Type | What it tests | Use it when | Skip it when | | --- | --- | --- | --- | | Unit | One function/class in isolation | Pure logic, algorithms, parsers, calculators | Code is mostly orchestration with few branches | | Integration | Multiple modules wired together, real DB or in-memory equivalent | API handlers, React components with state, service layers | You can get the same confidence from a unit test | | Contract | Producer and consumer agree on a schema (Pact, OpenAPI diff) | Microservices, public APIs, separate teams | Monolith with a single team | | End-to-end (E2E) | Full user flow through a real browser/app | Smoke-test the critical path: signup, checkout, search | You are tempted to test every edge case here — those belong lower | | Smoke | "Did the deploy come up?" — a tiny critical-path E2E | Every deploy, every environment | Never skip | | Regression | Old bugs do not come back | Add one whenever you fix a bug | You did not write a failing test before the fix | | Snapshot | Output matches a stored fixture | Stable, deterministic output: serializers, generators, public DTOs | UI components — they churn and snapshots become rubber-stamps | | Mutation | Your tests fail when code is deliberately broken | Critical modules, AI-generated test suites | You cannot afford 10–60x runtime — run nightly instead | | Property-based | Invariants hold for generated inputs | Parsers, sorts, encoders, anything with mathematical properties | Pure UI work | | Fuzz | Random/malformed input does not crash | Anything that parses untrusted bytes | Internal-only typed APIs | | Chaos | System survives infrastructure failures | Distributed systems with SLOs | Single-binary apps with no dependencies | | Load/performance | Latency and throughput under concurrent load | Anything user-facing with a perf SLO | Internal batch jobs measured by wall clock | | Security | OWASP Top 10, dependency CVEs, SAST/DAST | Always | Never — bake into CI | | Accessibility | WCAG compliance — axe-core in CI | Any consumer-facing UI | Internal admin tools (still nice to have) | Two opinionated calls: **Snapshot tests are dead for UI components, alive for serializers.** Jest snapshots locking your React tree produced a generation of engineers who hit "u" to update without reading the diff — a rubber stamp, not a test. Snapshots still work where the diff is small and meaningful: GraphQL schemas, generated SDKs, API response DTOs. **Coverage targets above 80% are mostly cargo-culting.** Going from 80% to 95% is expensive and gains little unless paired with mutation testing. ThoughtWorks' April 2026 Radar (Vol. 34) flags mutation testing — Stryker, PIT, cargo-mutants — as the way to "shift focus from how much code is executed to how much code is actually verified." ## Frameworks Per Ecosystem Pick boring tools your team already knows over novel tools nobody knows. | Ecosystem | Unit/Integration | E2E | Mutation | Property-based | Performance | | --- | --- | --- | --- | --- | --- | | JavaScript/TypeScript | Vitest (default), Jest 30 (legacy) | Playwright | Stryker | fast-check | k6, Artillery | | Python | Pytest | Playwright (Python bindings) | mutmut, Cosmic Ray | Hypothesis | Locust, k6 | | Go | `go test` + testify | Playwright via testcontainers | go-mutesting | gopter, native fuzz (`go test -fuzz`) | k6, vegeta | | Java/Kotlin | JUnit 5 + AssertJ | Playwright (Java) or Selenium | PIT (PITest) | jqwik | Gatling, JMeter, k6 | | Ruby | RSpec, Minitest | Capybara + Playwright | Mutant | Rantly | k6, JMeter | | Rust | Built-in `cargo test`, rstest | Playwright via webdriver | cargo-mutants (Trial on TW Radar) | proptest, quickcheck | criterion (microbench), k6 | | .NET | xUnit, NUnit | Playwright (.NET) | Stryker.NET | FsCheck | NBomber, k6 | **Vitest vs Jest:** for new projects, Vitest. 2026 benchmarks show Vitest 2.0 finishing 10,000 React component tests ~3.8x faster than Jest 30 with ~40% lower memory overhead, and watch reruns in hundreds of milliseconds instead of seconds. Vitest crossed 40M weekly downloads while Jest plateaued near 36M; Angular 21 made it default. Migration is mostly mechanical. Stay on Jest only if your suite is already fast enough. **Playwright vs Cypress:** Playwright. It overtook Cypress on every metric — 33M vs 6.5M weekly npm downloads, real WebKit support, free built-in sharding, ~290ms per action vs Cypress's ~420ms. Cypress's time-travel debugger is still the best in class, but not worth giving up cross-browser coverage and free parallelization. Healthy Cypress suites do not need to panic-migrate; new projects should pick Playwright. For Python, **Pytest** remains uncontested. For Go, the standard library plus `testify` covers ~95% of needs and Go 1.18+ ships a real fuzzer. For Java, JUnit 5 with AssertJ and Testcontainers is canonical. ## The AI-Augmented Testing Wave Two distinct things are happening, and they get conflated constantly. **First: AI generating unit and integration tests inside your IDE.** Cursor, Claude Code, and the broader agent crop write plausible tests against your codebase in seconds. Done well, this is useful — Claude Code reasons about test patterns across the whole repo and writes consistent fixtures. Done badly, you get tests that pass without asserting anything meaningful: an LLM that mocks the function under test, then asserts the mock was called. The mitigation is mutation testing on AI-generated suites and human review focused on the assertions. See our deep-dives on [AI coding agents](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/), [Cursor](https://codersera.com/blog/cursor-ide-complete-guide-2026/), and [Claude Opus 4.7](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/). **Second: AI-native test platforms — Mabl, Functionize, ProductScript, QA Wolf, Autonoma.** These record or describe a flow in natural language, then maintain locators when the UI shifts. The State of Testing 2026 report shows AI-augmented tools delivering a 12.1% increase in automation coverage, ~10.8% drop in production defects, and 40–45% maintenance cost reductions on self-healing suites. Real numbers — but achieved by mature teams. Bolting Mabl onto a chaotic codebase will not save it. Rough 2026 pricing: **Mabl** ~$40k–$80k/year; **Functionize** enterprise from $50k+; **QA Wolf** managed service $5k–$30k/month; **ProductScript** and similar AI-agent tools $200–$2,000/month per team. Decision rule: if your engineers will write and maintain tests, stay on open-source (Playwright + Vitest/Pytest) and let AI assist in the IDE. If they will not write tests under any circumstances, an AI-native platform beats the zero-test status quo — but it is a stopgap, not a strategy. ## CI/CD Test Orchestration The biggest CI improvement most teams skip is **test impact analysis** — only running tests affected by the diff. Datadog Test Optimization, Bazel, Nx, and Turborepo all do this. Faster PRs, less flake exposure, lower CI bills. A working 2026 CI pipeline for a typical web app: 1. **On every push:** lint, typecheck, unit tests for changed modules (test impact analysis), security scans (Snyk, Dependabot, Trivy for containers). 2. **On PR:** integration tests, contract tests against a shared mock provider (Pact Broker / PactFlow), accessibility scan with axe-core, full Playwright suite sharded across 4–8 workers. 3. **On merge to main:** deploy to a staging or ephemeral preview environment, run smoke E2E, gate on a k6 or Artillery perf check that asserts p95 latency hasn't regressed beyond a threshold. 4. **Nightly:** mutation testing run, full E2E across all browsers, dependency audit, chaos experiments in non-prod. Run the slow tests where they belong — at night, not blocking PRs. A 45-minute PR pipeline trains the team to skip tests; a 6-minute PR pipeline trains the team to write more. ## Flaky Tests: Quarantine, Do Not Retry Flaky tests cost the average engineering team 6–8 hours per engineer per week, according to Datadog's 2026 telemetry. The wrong response is automatic retries. Retries hide real bugs, normalize unreliability, and exhaust CI budget. The right response is a quarantine workflow: 1. Detect flakes statistically — same commit, both pass and fail across runs. Datadog, Trunk.io, BuildPulse, and CircleCI all surface this now. 2. Auto-quarantine flaky tests so they stop blocking CI but still report. 3. File a ticket with full context and an owner. 4. Set a hard SLA — a quarantined test that is not fixed in two weeks gets deleted, not ignored. Datadog's Bits AI Dev Agent now auto-generates fixes for detected flakes as PRs; we have seen it produce solid fixes for race conditions and selector instability and weak fixes that just paper over async timing. Treat those PRs like any AI-generated PR: the diff matters, the green check does not. ## Performance Testing The four mainstream tools: - **k6:** Go binary, JS scripting, lowest CPU/memory per VU, best CI/CD integration. Free; Grafana Cloud k6 from ~$30/month. - **Locust:** Python-native, real-time UI, friendly to AI-assisted script generation. Pick this for Python-first teams. Free. - **JMeter:** broadest protocol support (JDBC, LDAP, JMS, SMTP). XML plans fight version control; for non-HTTP protocols nothing else competes. Free. - **Artillery:** YAML-driven; GraphQL, gRPC, WebSockets, Kafka, Playwright browser load. Free core, paid cloud. Default to k6 for HTTP/gRPC perf in CI, Locust for Python shops, JMeter only when you need its protocols. Run perf tests against ephemeral envs, not shared staging. ## Test Environments and Data The 2026 default is **ephemeral preview environments** — a full stack spun up per PR, torn down on merge. Vercel and Netlify ship this for static/BFF workloads; Bunnyshell, Northflank, Shipyard, and Tilt's ephemerator handle Kubernetes. Garden remains strongest for declarative multi-service envs wired into local dev. Shared staging is a tragedy of the commons — conflicting changes, drifting data, "is this broken?" Slack threads as the actual gate. Ephemeral envs cost more in cloud bill and pay back in test reliability. Test data is the other half. Three patterns that work: - **Builders/factories** (factory-bot, fishery, polyfactory) for unit and integration tests — readable, composable, no test pollution. - **Sanitized production snapshots** for performance and integration testing — Tonic, Snaplet, Neosync, or in-house pipelines. Real shapes, no real PII. - **Reset-per-test database transactions** for integration tests against a real DB — Postgres `TRUNCATE` is fine; transactional rollback is faster but fights with code that opens its own transactions. ## Shift-Left, Shift-Right, and Observability-as-Testing Shift-left — testing earlier in the cycle — has been the consensus for a decade. The newer movement is shift-right: testing in production with feature flags, canary deploys, synthetic monitors, and structured observability. Datadog, Honeycomb, and Sentry have collapsed the line between "test" and "monitor." If your synthetic checks fire k6 scripts every five minutes against production with budgeted error rates, that is testing. If your deploys are staged behind feature flags with automatic rollback on error-rate regression, that is testing. The 2026 reality: fewer pre-prod tests, more guardrails around production. The marginal hour spent on a 99th E2E test is usually worse-spent than wiring up a synthetic check, SLO burn-rate alert, or feature-flag rollback. Chaos engineering belongs here too. Tools to know: - **LitmusChaos** — CNCF, Kubernetes-native, the open-source default. - **Gremlin** — commercial, polished UI, the safest bet for first-time chaos. - **Chaos Mesh** — CNCF, strong network and IO fault injection. - **AWS Fault Injection Service** — if you live in AWS, the lowest-friction option. Chaos Monkey itself is mostly historical now — useful as a reference, not a tool you would deploy fresh in 2026. ## When to Outsource QA, When to Hire an SDET, When to Just Hire Better Engineers The honest answer most engineering leaders do not want to hear: most teams asking "should we hire a QA team?" should instead hire engineers who write tests as part of shipping. A separate QA team becomes a wall to throw quality over, and engineering's testing skill atrophies. Where dedicated QA capacity does make sense: - **Regulated industries** — finance, healthcare, automotive — where audit trails and signed-off test plans are mandatory. - **Complex hardware/software products** with lab environments or specialized devices (see our [Android emulators guide](https://codersera.com/blog/android-emulators-complete-guide-2026/)). - **Mature products at scale** where one SDET owns E2E infrastructure, flake triage, and test tooling. - **Outsourced QA pods** for surge work — launches, large UI overhauls — never as a permanent crutch. For most early- and mid-stage teams, the highest-leverage move is hiring engineers whose definition of "done" includes tests, observability, and deploy safety. That is what Codersera screens for. ## Sharp Edges and Common Mistakes - **Mocking the thing you are testing.** If your test mocks the function under test and asserts the mock was called, you have written a tautology. Common with AI-generated tests; catch it with mutation testing or careful review. - **Coverage as a goal, not a signal.** 90% line coverage with weak assertions is worse than 60% with strong ones — it manufactures false confidence. Pair coverage with mutation scoring before celebrating. - **One giant E2E suite as the only test.** Slow, flaky, expensive, and tells you "something broke" without telling you what. Push tests down the pyramid/trophy. - **Auto-retrying flaky tests.** Hides real concurrency bugs and trains the team that green CI does not mean working code. - **Snapshot-spamming UI tests.** Updating snapshots without reading the diff is not a test, it is a habit. Use snapshots only where the diff is small and meaningful. - **Performance tests against shared staging.** Misleading results, angry teammates. Spin up an ephemeral env per perf run. - **Skipping accessibility.** axe-core in CI takes one afternoon to set up and catches most WCAG issues. ThoughtWorks moved it to "Adopt" for a reason. - **Mutation testing on the wrong scope.** 10–60x slower than your test suite; run on critical modules nightly, not on every PR. - **Treating tests as second-class code.** Test code with no review, no refactoring, and no DRYing decays into the worst code in your repo. Review tests like production code. - **Letting AI-generated tests through without review.** They look right, they run green, and they assert nothing. The 30 seconds of review you skip costs you a week of debugging later. ## FAQ ### Is the test pyramid dead? No. It is just no longer universal. For backend services with rich domain logic, the pyramid is still right. For frontends and orchestration code, the testing trophy is the better mental model. Pick deliberately. ### What code coverage target should we hit? For most production code, 70–80% line coverage with strong assertions and a mutation score above 60% is solid. Above 90% line coverage typically signals diminishing returns and brittle tests. Critical modules — auth, billing, anything money-touching — earn 90%+ and a mutation score above 80%. ### Should we migrate from Jest to Vitest? For new projects, yes — Vitest by default. For existing Jest projects, migrate when the suite is slow enough that engineers avoid running it locally. Mechanical migrations of mid-size suites usually take 1–3 days; transformer-heavy projects take longer. ### Should we migrate from Cypress to Playwright? Only if you need cross-browser (Safari/WebKit) coverage, hit Cypress Cloud parallelization paywalls, or are starting fresh. Healthy Cypress suites are not worth disrupting just for the badge. ### Are AI-generated tests safe to merge? Only after human review focused on the assertions. The structure looks right by default; the assertions are where AI-written tests fail silently. Run mutation testing on suites that include heavy AI contributions. ### How do we deal with flaky tests? Detect statistically, quarantine fast, file a ticket with an owner, hard SLA on fixing or deleting. Never auto-retry blindly. ### Is contract testing worth setting up? Yes, if you have multiple services owned by different teams and you want to retire most cross-service E2E tests. Pact with PactFlow is the canonical setup; budget two weeks for the first integration. ### Is mutation testing too slow to use? For full-suite use, yes — 10–60x slower than your tests. Scope it: nightly runs on critical modules, or on PRs that touch high-stakes code paths. Stryker, PIT, and cargo-mutants all support module-level scoping. ### k6 or Locust for performance testing? k6 if you want lower resource use and the best CI integration. Locust if your team is Python-first and you want to share helpers with your app. Both are excellent. ### Should we run chaos engineering experiments in production? Eventually, yes — Netflix's whole point. Start in staging or ephemeral envs with LitmusChaos or Gremlin, build confidence, then graduate to production with strict blast-radius controls and clear stop conditions. ### What is observability-as-testing? The idea that production telemetry — error rates, SLOs, synthetic monitors, feature-flag-gated rollouts with automatic rollback — does some of the work that pre-prod tests used to. Not a replacement for tests, but a complement that catches what pre-prod cannot. ### When does it make sense to hire a dedicated SDET? When test infrastructure (E2E platforms, flake triage, perf harnesses, environment management) is full-time work, typically around the 8–15 engineer mark for product teams, or earlier for regulated and high-uptime products. ### Should we outsource QA? For surge work — launches, large UI overhauls, regression hardening — yes, an outsourced pod can be efficient. As a permanent function for an engineering team that should be writing tests itself, no. It tends to ossify the "engineering writes code, QA finds bugs" anti-pattern. ### What is the single highest-ROI testing investment for a 10-engineer team in 2026? Test impact analysis plus ephemeral preview environments. Together they cut PR-cycle time, reduce flake exposure, and let real integration tests run on every change without team contention. ## Next Steps Testing is not a phase of the SDLC, it is part of the discipline of writing software. The teams shipping reliably in 2026 are the ones whose engineers internalized that — they pick the right test type for the job, they treat AI-generated tests with healthy suspicion, they invest in CI infrastructure (test impact analysis, ephemeral envs, flake quarantine) before they invest in headcount, and they treat production observability as part of the test loop. If you are hiring engineers and the candidates you are seeing treat testing as someone else's job, you have a hiring problem, not a process problem. [Hire a Codersera-vetted engineer who treats testing as part of the job](https://codersera.com/blog/hire) — and keep the QA-team-shaped hole on your org chart empty until you actually need it.

# Codersera — AI engine knowledge base (llms-full.txt) # Generated: 2026-08-13T20:02:12Z # Source: https://codersera.com/blog// per pillar # 20 pillar guides covering AI models, developer tooling, mobile testing # License: human-readable text excerpts under fair use; full HTML at the URLs above # Regenerate: yarn build:llms (script at scripts/build-llms-full.js) ## Table of contents - Cluster: AI / models 1. [DeepSeek V4: The Complete Guide (2026)](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) 2. [Claude Opus 4.7: The Complete Developer Guide (2026)](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/) 3. [GPT-5.5: The Complete Developer Guide (2026)](https://codersera.com/blog/gpt-5-5-complete-guide-2026/) 4. [Llama 4: The Complete Developer Guide (2026)](https://codersera.com/blog/llama-4-complete-guide-2026/) 5. [Qwen 3.5: The Complete Developer Guide (2026)](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) 6. [Gemma 4: The Complete Developer Guide (2026)](https://codersera.com/blog/gemma-4-complete-guide-2026/) 7. [Kimi K2.6: The Complete Developer Guide (2026)](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/) 8. [The Open-Source LLM Landscape in 2026](https://codersera.com/blog/open-source-llms-landscape-2026/) 9. [Gemini 3.5 Complete Guide: Flash, Pro, Pricing & How It Compares (2026)](https://codersera.com/blog/gemini-3-5-complete-guide-2026/) - Cluster: Developer tooling 10. [AI Coding Agents in 2026: The Complete Guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) 11. [AGENTS.md and SKILL.md: The Complete Guide (2026)](https://codersera.com/blog/agents-md-complete-guide-2026/) 12. [Cursor IDE in 2026: The Complete Developer's Guide](https://codersera.com/blog/cursor-ide-complete-guide-2026/) 13. [Self-Hosting LLMs in 2026: The Complete Guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) 14. [Apple Silicon LLMs: Complete Guide to Running Models on Mac (2026)](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) 15. [Fine-Tuning LLMs: The Complete 2026 Guide (LoRA, QLoRA, Unsloth, Axolotl, MLX)](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) 16. [Self-Training a Small LLM From Scratch: The 2026 Complete Guide](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) - Cluster: Mobile & testing 17. [Android Emulators in 2026: The Complete Guide for Developers](https://codersera.com/blog/android-emulators-complete-guide-2026/) 18. [iOS Simulators in 2026: The Complete Guide for Developers](https://codersera.com/blog/ios-simulators-complete-guide-2026/) 19. [Mobile App Testing in 2026: The Complete Guide](https://codersera.com/blog/mobile-app-testing-complete-guide-2026/) 20. [Software Testing in 2026: The Complete Engineer's Guide](https://codersera.com/blog/software-testing-complete-guide-2026/) =============================================================== [1/20] DeepSeek V4: The Complete Guide (2026) URL: https://codersera.com/blog/deepseek-v4-complete-guide-2026/ Last updated: 2026-08-13 =============================================================== *Last updated: August 13, 2026. We refresh this guide whenever DeepSeek ships a V4 patch, the major IDEs change their integration, or the pricing moves.* **Quick answer.** The DeepSeek V4 API offers two models: **DeepSeek V4 Pro** at $0.435 / 1M input and $0.87 / 1M output (cache-hit input $0.003625), and **DeepSeek V4 Flash** at $0.14 / $0.28 (cache-hit $0.0028). **V4-Pro reached general availability on August 13, 2026** (checkpoint V4-Pro-0813); these rates have been the standing pricing since 2026-05-22 and hold until **16:00 UTC on August 16, 2026**, when both models move to higher peak/off-peak tiers — making V4 Pro roughly 34× cheaper on input and 86× cheaper on output than Claude Opus 4.7, with weights MIT-licensed for self-hosting. Source: [api-docs.deepseek.com](https://api-docs.deepseek.com/quick_start/pricing), [HN 48237663](https://news.ycombinator.com/item?id=48237663). **DeepSeek V4** is the open-weights model that quietly reset the price-performance frontier when it shipped on April 24, 2026. Two variants — **V4 Pro** (1.6T total parameters, 49B active) and **V4 Flash** (284B total, 13B active) — both ship with a 1M token context window, MIT licensing, and pricing that — permanent since DeepSeek's 2026-05-22 announcement — lands V4 Pro at roughly 1/34th the cost of Claude Opus 4.7 on input (1/86th on output) and ~1/11th the input cost of GPT-5.5 on coding workloads. This page is the single landing surface we point engineering teams to when they need to evaluate, deploy, or migrate to V4. Every section links to a deep-dive when you need to go further. ## TL;DR — Should you care? - **If you ship code with an LLM in the loop:** yes. V4 Pro tops LiveCodeBench at 93.5, hits Codeforces ELO 3206 (ahead of GPT-5.5 at 3168), and is statistically tied with Claude Opus 4.7 on SWE-bench Verified (80.6 vs 80.8) — at a small fraction of the cost. - **If you self-host:** V4 Flash is the practical target. Native FP4+FP8 mixed quantization runs on a single H100 (tight) or comfortably on 4×H200. V4 Pro requires 8×H100 minimum at FP8. - **If you're paying Anthropic or OpenAI for coding agents:** the migration is a one-flag change for OpenAI-SDK clients. **DeepSeek made the 75% V4 Pro discount permanent on May 22, 2026** — V4 Pro now sits at $0.435/$0.87 per 1M tokens in/out as the published list price, roughly 35× cheaper on input and 86× cheaper on output than Claude Opus 4.7. Start the eval now. - **If you are still treating V4 as a preview:** it is not one. **V4-Pro went GA on August 13, 2026** with the **V4-Pro-0813** checkpoint — near-identical architecture plus [DSpark speculative decoding](https://codersera.com/blog/deepseek-dspark-explained-2026/), reasoning-effort levels (`low` / `high` / `max`) on both models, a native OpenAI Responses API with one-click Codex setup, and published concurrency limits (500 on Pro, 2,500 on Flash). Model names unchanged; inference behaviour changed, so re-run your evals. [Full 0813 guide](https://codersera.com/blog/deepseek-v4-pro-0813-guide-2026/). - **Pricing rises on August 16, 2026:** from 16:00 UTC both models move to peak/off-peak tiers. Off-peak is not cheaper than today — it is half of a raised peak. [Breakdown here](https://codersera.com/blog/deepseek-v4-price-change-august-2026/). - **The catch:** the multi-turn `reasoning_content` 400 error is real and breaks every popular client on first contact. Fix exists; details below. - **Heads up:** the legacy `deepseek-chat` and `deepseek-reasoner` aliases retire **July 24, 2026 15:59 UTC**. If your code hard-codes them, plan the cutover. ## What is DeepSeek V4? DeepSeek V4 is the fourth-generation flagship from DeepSeek AI, released as a preview on April 24, 2026. It supersedes the V3.2-Exp branch and effectively replaces the R1 reasoning line — V4's optional thinking mode covers what R1 was used for. There are two production variants plus base-model variants for fine-tuning: | Variant | Total params | Active per token | Context | License | Best for | | --- | --- | --- | --- | --- | --- | | **V4 Pro** | 1.6T | 49B | 1M tokens | MIT | Frontier coding agents, complex reasoning, long-horizon tasks | | **V4 Flash** | 284B | 13B | 1M tokens | MIT | Cheap inference, IDE integration, high-throughput batch, self-hosting | If you're choosing between them, the [Pro vs Flash comparison](https://codersera.com/blog/deepseek-v4-pro-vs-flash/) walks through cost-per-task math on real workloads. For the official release breakdown, see [DeepSeek V4: Full Release Breakdown](https://codersera.com/blog/deepseek-v4-release-date-features-benchmarks/) and [V4 Is Here: Full Specs, Benchmarks, and API Guide](https://codersera.com/blog/is-deepseek-v4-released-official-status-specs-alternatives/). ## Architecture: what changed vs V3.2 V4 keeps the Mixture-of-Experts backbone DeepSeek has been refining since V2 but introduces three load-bearing changes: 1. **DeepSeek Sparse Attention (DSA).** A hybrid of Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA). Net effect at 1M context: roughly **27% of the per-token FLOPs** and **10% of the KV cache footprint** versus V3.2. This is what makes a 1M context window economically viable rather than a marketing bullet. 2. **Three reasoning modes.** Non-Think, Think High, Think Max. Default is Think enabled. Unlike R1, V4 supports tool calls in thinking mode — previously you had to choose between reasoning and tools. 3. **Manifold-Constrained Hyper-Connections (mHC)** for training stability. Mostly relevant if you're fine-tuning; doesn't change inference behavior. If you're trying to decide whether to upgrade an existing V3.2 deployment, the [V4 vs V3.2 changes guide](https://codersera.com/blog/deepseek-v4-vs-deepseek-v32-what-changed-developers-should-use/) walks through the breaking changes (mostly API-level: `reasoning_content` behavior, prefix cache hashing). For the family-wide view across the whole DeepSeek lineup, the [V3 vs V4 deep dive](https://codersera.com/blog/deepseek-v3-vs-deepseek-v4-a-deep-dive-into-ai-innovation-and-performance/) covers architecture, benchmarks, and pricing side-by-side. ## Benchmarks (the real numbers) Numbers below are from DeepSeek's official model card on HuggingFace, cross-validated against ArtificialAnalysis where independent evals exist. AIME, MATH-500, and IFEval are *not* in the official card and we've omitted them rather than quote secondary sources. | Benchmark | V4 Pro (Max) | Claude Opus 4.7 | GPT-5.5 | Gemini 3.1 Pro | | --- | --- | --- | --- | --- | | SWE-bench Verified | 80.6 | 80.8 | — | — | | SWE-bench Pro | 55.4 | 64.3 | 58.6 | 54.2 | | LiveCodeBench | **93.5** | 88.8 | — | 91.7 | | Codeforces ELO | **3206** | — | 3168 | — | | GPQA Diamond | 90.1 | 94.2 | 93.6 | — | | MMLU-Pro | 87.5 | — | — | — | | GSM8K | 92.6 | — | — | — | | TerminalBench 2.0 | 67.9 | — | — | — | | BrowseComp | 83.4 | 79.3 | 84.4 | — | | SimpleQA-Verified | 57.9 | — | — | 75.6 | | MRCR (1M long-context) | 83.5 | — | — | — | | CorpusQA (1M long-context) | 62.0 | — | — | — | The honest read: **V4 Pro leads on coding-style benchmarks (LiveCodeBench, Codeforces) and is essentially tied with Claude Opus 4.7 on SWE-bench Verified.** It loses on pure factual recall (SimpleQA-Verified, where Gemini 3.1 Pro dominates) and on hard multi-step reasoning (GPQA Diamond, where Claude leads). On SWE-bench Pro — the harder agent-style coding eval — V4 Pro is meaningfully behind Claude Opus 4.7 (55.4 vs 64.3). For long-horizon, tool-heavy agentic loops, Claude is still the more reliable choice; for raw coding velocity at scale, V4 Pro wins. **Independent harnesses, August 2026 (the numbers above are vendor-reported on the preview).** The verdict on the GA checkpoint splits sharply by harness. On Vals' neutral harness, **V4-Pro-0813 places 2nd on SWE-bench Verified at 96.40% (±0.83)** — behind only Claude Opus 5 (97.00%), ahead of GPT-5.6 Sol (96.20%) and Grok 4.6 (95.60%) — at **$0.022 per test versus Grok's $0.785 and Opus 5's $1.29**. DeepSeek never claimed a SWE-bench number, so that result is purely independent. But on Terminal-Bench, DeepSeek claims 87.9% on its own unreleased harness while the reference Terminus 2 harness scores V4-Pro at **54.68%** — a 33-point gap, and 12 points *below* its own cheaper sibling V4-Flash (67.04%). Every lab drops 7–13 points moving to Terminus 2, so Pro's fall is a categorical outlier rather than harness noise. On LiveBench agentic coding, both DeepSeek models rank last of seven frontier peers (Pro 54.95, Flash 46.77 vs Opus 5's 65.20). And on AA-Omniscience — does the model know what it does not know — **V4-Pro-0813 scores 0.83 against Opus 5's 37.07**, near the hallucination floor. Read V4 as a coding-benchmark leader with a real reliability weakness, not an all-round frontier model. For deeper comparisons see [DeepSeek V4 Pro Review](https://codersera.com/blog/deepseek-v4-pro-review-benchmarks-pricing-2026/), [V4 vs Claude Opus 4.7 head-to-head](https://codersera.com/blog/deepseek-v4-vs-claude-opus-4-7/), [V4 vs GPT-5.5 and GPT-5.5 Pro](https://codersera.com/blog/deepseek-v4-vs-gpt-5-5-pro/), and the broader [V4 vs Qwen, Kimi, MiniMax, GPT, Claude comparison](https://codersera.com/blog/deepseek-v4-alternatives-qwen-kimi-minimax-gpt-claude-compared/). ## Pricing API pricing on api.deepseek.com (USD per 1M tokens). DeepSeek announced on **May 22, 2026** that the 75% V4 Pro discount originally scheduled to expire on May 31 is now the permanent standard list price — the previous "launch promo" wording is retired. V4 Flash pricing is unchanged from launch: | Model | Cache hit (in) | Cache miss (in) | Output | | --- | --- | --- | --- | | V4 Pro | $0.003625 | $0.435 | $0.87 | | V4 Flash | $0.0028 | $0.14 | $0.28 | | V3.2-Exp (legacy) | $0.027 | $0.27 | $1.10 | **This changes at 16:00 UTC on August 16, 2026.** DeepSeek splits both models into peak (01:00–04:00 and 06:00–10:00 UTC) and off-peak tiers. Off-peak is *not* a discount on today’s price — it is half of a raised peak, and every tier bills above the current flat rate: | Model / tier (from 2026-08-16) | Cache hit (in) | Cache miss (in) | Output | | --- | --- | --- | --- | | V4 Pro — off-peak | $0.022 | $0.66 | $1.98 | | V4 Pro — peak | $0.044 | $1.32 | $3.96 | | V4 Flash — off-peak | $0.007 | $0.22 | $0.66 | | V4 Flash — peak | $0.014 | $0.44 | $1.32 | Blended that is roughly **1.96x on input, 2.94x on output and 7.84x on cache reads**. The cache-read multiplier is the one that stings: it lands hardest on agent loops with a large stable prefix, which is exactly where V4 is cheapest today. Even at peak, V4 Pro stays an order of magnitude under Opus 4.7 — but the headline multiplier drops from ~86x to roughly 19–38x on output. [Full working here](https://codersera.com/blog/deepseek-v4-price-change-august-2026/). For comparison: Claude Opus 4.7 is **$15 / $75 per million** in/out, GPT-5.5 is **$5 / $30**. At the new permanent list price V4 Pro is roughly **35× cheaper on input and 86× cheaper on output** than Opus 4.7. Neither of those ratios is a typo — and they're no longer time-limited. The cache-hit price is the load-bearing detail. **DeepSeek dropped the cache-hit rate to 1/10 of launch pricing on April 26, 2026**, which means agentic loops with stable system prompts get charged at one-tenth the listed input rate. Well-structured coding agents typically see >70% cache hit rates. For end-to-end pricing math including realistic cache-hit modeling, see [How to Use DeepSeek V4 API: Complete Developer Guide](https://codersera.com/blog/how-to-use-deepseek-v4-api-developer-guide-2026/). ## Hardware requirements (self-hosting) Numbers below assume vLLM 0.20.0+ with V4's native FP4+FP8 mixed quantization (FP4 for experts, FP8 for attention/norm/router). Community AWQ INT4 builds for V4 Flash exist with ~5% quality loss but aren't the official recipe. | Hardware | V4 Flash | V4 Pro | Notes | | --- | --- | --- | --- | | RTX 4090 / 5090 (24–32GB) | Doesn't fit | — | Even Flash needs ~158GB at native quant | | Single H100 (80GB) | INT4 only, tight | Doesn't fit | Use AWQ build; expect ~5% quality loss vs native | | Single H200 (141GB) | FP4+FP8 native, tight | Doesn't fit | Recommended dev target for Flash | | 2× H100 (160GB) | FP8 native, comfortable | Doesn't fit | Cleanest small-scale Flash deploy | | 4× H200 / GB200 NVL4 tray | **Official vLLM recipe** | Doesn't fit | Production Flash target | | 8× H100 (640GB NVLink) | Overkill | FP8 minimum, capped ~800K context | Production Pro entry point | | 8× H200 / DGX H200 | Overkill | **Official vLLM recipe**, full 1M ctx | Recommended Pro deploy | | 8× B200 / B300 | Overkill | Optimal — DP+EP | Highest throughput single-node | | RTX PRO 6000 Blackwell | **Broken** (vLLM #40821) | Broken | SM120 unsupported in vLLM compile path | | AMD MI325X | Community WIP | Community WIP | No official recipe yet | | Mac M3 Ultra 192GB (MLX) | 4-bit Flash works | No | Inference only; ~14 min prompt processing for 8K tokens on llama.cpp fork | The practical version: **V4 Flash on 4×H200 (or a single H200 for dev) is the answer for most self-hosted production workloads.** Pro is for teams that already have a multi-GPU node and want frontier quality on tap. **Avoid RTX PRO 6000 entirely** until the vLLM Inductor compile-path issue is resolved — it crashes on load. For the full step-by-step on bringing Flash up locally — including the vLLM config, FP4+FP8 quantization, and the launch flag set — follow [Run DeepSeek V4 Flash Locally: Full 2026 Setup Guide](https://codersera.com/blog/run-deepseek-v4-flash-locally-full-2026-setup-guide/). For the architectural deep-dive on why Flash is the right choice for most teams, see [DeepSeek V4 Flash: The Practical Deep Dive](https://codersera.com/blog/deepseek-v4-flash-deep-dive/). ## API integration DeepSeek exposes both an OpenAI-compatible base URL (`https://api.deepseek.com`) and an Anthropic-compatible base URL (`https://api.deepseek.com/anthropic`). For most teams, OpenAI compat is the path of least resistance — your existing SDK works with two changes: ``` from openai import OpenAI client = OpenAI( api_key=os.environ["DEEPSEEK_API_KEY"], base_url="https://api.deepseek.com/v1", ) resp = client.chat.completions.create( model="deepseek-v4-pro", # or "deepseek-v4-flash" messages=[ {"role": "system", "content": "You are an expert Python engineer."}, {"role": "user", "content": "Refactor this function to be O(n)..."}, ], extra_body={"thinking": "high"}, # "off", "high", or "max" ) print(resp.choices[0].message.content) print(resp.choices[0].message.reasoning_content) # may be populated ``` JSON mode (`response_format={"type": "json_object"}`), tool calling (`tools=[...]` with optional strict mode), and SSE streaming all work as expected. **V4 supports tool calls in thinking mode** — this was R1's biggest limitation and is now fixed. ### The `reasoning_content` 400 error This is the single biggest gotcha and the one that breaks every popular client on first contact. Worth understanding in detail. When V4 returns a response with thinking mode enabled, the message object contains both `content` (final answer) and `reasoning_content` (the chain-of-thought). On the *next* request in the same conversation, you must round-trip `reasoning_content` back as part of the assistant message. If you only pass `content`, the API responds with `400 Bad Request: "reasoning_content must be passed back"`. This is the *opposite* of how R1 worked. R1 rejected requests that included `reasoning_content` on prior turns. V4 requires it. Most clients still hard-code R1's behavior: - **LiteLLM** — [issue #26395](https://github.com/BerriAI/litellm/issues/26395); strips `reasoning_content` when serializing message history. Fix is a one-line patch. - **OpenCode** — [#24190](https://github.com/anomalyco/opencode/issues/24190), [#24722](https://github.com/anomalyco/opencode/issues/24722); documented Anthropic-API workaround. - **OpenClaw / Kilo Code / Hermes-agent** — same root cause, separate issue trackers. - **Cursor** — breaks intermittently on long sessions; community proxy `deepseek-cursor-proxy` works around it. There's also a tool-call wrinkle: even on assistant turns where there was no thinking (e.g., a pure tool call), some clients need to include `reasoning_content: ""` (empty string, not `null`) to satisfy V4's validator on the next turn. Our [complete API developer guide](https://codersera.com/blog/how-to-use-deepseek-v4-api-developer-guide-2026/) walks through the proxy pattern that fixes this once and for all. ### Model names on the API - `deepseek-v4-pro` — V4 Pro (current alias, stable) - `deepseek-v4-flash` — V4 Flash (current alias, stable) - `deepseek-chat` — legacy alias, currently routes to V4 Flash non-thinking mode. **Retires July 24, 2026 15:59 UTC.** - `deepseek-reasoner` — legacy alias, currently routes to V4 Flash thinking mode. **Retires July 24, 2026 15:59 UTC.** **Heads up:** if your code hard-codes `deepseek-chat` or `deepseek-reasoner`, plan the cutover before July 24. Pricing already changed — many users were silently upgraded from V3.2 to V4 Flash on April 24. ## Local deployment options | Stack | V4 Flash | V4 Pro | Notes | | --- | --- | --- | --- | | vLLM 0.20.0+ | Production-ready (official recipe) | Production-ready (official recipe) | The reference deployment. Required flags: `--trust-remote-code --kv-cache-dtype fp8 --enable-expert-parallel --tokenizer-mode deepseek_v4 --reasoning-parser deepseek_v4` | | SGLang 0.4+ | Production-ready | Production-ready | Better burst / short-request throughput than vLLM on agentic loops | | TGI (HuggingFace) | Beta | Beta | 1M context not officially supported yet | | Ollama | Experimental | No | llama.cpp MoE routing not yet optimal; quality regressions vs vLLM | | LM Studio | Yes | No | Single-user; not for production | | MLX (Mac) | 4-bit Flash works on M3 Ultra 128GB+ | No | Inference only; 192GB recommended for headroom | | llama.cpp | Experimental fork (antirez/llama.cpp-deepseek-v4-flash) | No | Targets MacBook 128GB w/ 2-bit experts | **One sharp edge worth flagging:** there is no standard HuggingFace Jinja chat template for V4. Naive tokenizer pipelines that assume one will silently produce malformed prompts. Use the encoding scripts that ship in each model's HuggingFace repo, not generic chat templates. For most teams the choice is vLLM (familiar, broad ecosystem, official recipes) or SGLang (faster on coding-agent workloads where long prefix caches dominate). Our [local setup guide](https://codersera.com/blog/run-deepseek-v4-flash-locally-full-2026-setup-guide/) walks through the vLLM path end-to-end with the correct flags. ## DSpark: 51–400% faster inference (June 2026 update) On June 27, 2026, DeepSeek released **DSpark** — an open-source speculative-decoding module that attaches to the unchanged V4 checkpoints and lifts decoding throughput by **51–400%** versus the built-in single-MTP path, with identical output quality. Both variants ship pre-bundled on Hugging Face as `DeepSeek-V4-Pro-DSpark` and `DeepSeek-V4-Flash-DSpark` (MIT license), alongside the [DeepSpec](https://github.com/deepseek-ai/DeepSpec) training/eval framework. DeepSeek also published ready-trained DSpark draft checkpoints for Qwen3 (4B/8B/14B) and Gemma-4-12B-it, so the speedup isn’t limited to DeepSeek’s own models. If you self-host V4, the DSpark variant is now the default choice — full details in our [DSpark explainer](https://codersera.com/blog/deepseek-dspark-explained-2026/). (Not to be confused with NVIDIA’s DGX Spark hardware.) ## IDE and tool integration V4 works in every popular agentic-coding IDE; each one has its own integration sharp edge tied to the `reasoning_content` handshake. - **Claude Code** — official integration via the Anthropic-compat base URL. DeepSeek publishes a dedicated Claude Code guide. The pitch is now permanent: V4 Pro is roughly 1/34 the input cost and 1/86 the output cost of Opus 4.7 on the same Claude Code workflows. - **Cursor** — custom model via OpenAI-compat (`Cmd+,` → Models). Long sessions break on the multi-turn 400; community proxy `deepseek-cursor-proxy` patches it. - **Cline / Roo Code / Kilo Code** — native DeepSeek provider. Disable thinking mode for now or pin to a build with the round-trip fix. - **Aider** — works out of the box: `aider --model deepseek/deepseek-v4-pro`. - **Continue.dev** — OpenAI-compatible config. Same multi-turn caveat as Cursor. - **OpenCode / OpenClaw / Hermes-agent** — all currently affected by the 400 error in multi-turn agent loops. Watch each project's tracker for May 2026 patches. ## Where V4 actually shines ### Agentic coding loops V4 Pro's combination of high LiveCodeBench (93.5), Codeforces 3206, tool-calls in thinking mode, and ~1/34 Opus input pricing (~1/86 on output) makes it the first frontier model where running a long-horizon coding agent is economically rational. At Claude Opus 4.7 prices an 8-hour autonomous run typically costs $50–200; the same workload on V4 Pro at its permanent list price lands at roughly $0.60–2.40, and stays there — no "promo ends" cliff to plan around. ### Codebase Q&A without RAG V4's MRCR 1M score of 83.5 and CorpusQA 1M of 62.0 are strong enough that fitting a whole service-sized codebase (~750K tokens) into a single prompt is a viable replacement for a maintained RAG pipeline. Cache-hit pricing ($0.003625 / 1M for V4-Pro, $0.0028 / 1M for V4-Flash) makes re-querying the same repo prefix essentially free at routine workflow volumes. ### Long-document analysis Legal review, log triage, multi-hour transcript analysis — anywhere the prior pattern was "summarize then re-summarize," V4 lets you keep the full corpus in context. Cache the document, query repeatedly cheaply. ### Fine-tuning NVIDIA shipped an official NeMo AutoModel recipe for V4 Flash on Blackwell GPUs on April 25, 2026 — handles V4-specific Q-LoRA / O-LoRA quirks. Pro fine-tuning needs a multi-node setup; most teams should fine-tune Flash and use Pro as a teacher. ### High-throughput batch generation V4 Flash at $0.14/$0.28 makes it the cheapest viable model for batch workloads (synthetic data, classification at scale, content reformatting). Self-hosted Flash on a 4×H200 node hits the API breakeven point at roughly 100M input tokens / month. ## Known issues and limitations - **Multi-turn thinking-mode 400 error** (described above). Real production hazard. - **vLLM #40821 / #41027 / #26211** — RTX PRO 6000 Blackwell (SM120) crashes on V4 load due to a torch.compile / Inductor `auto_functionalized` assertion. No fix shipped. Avoid this hardware until resolved. - **Server busy (HTTP 503/429) at peak hours** on the official API. Rate limits are dynamic and unpurchasable; production users front the API with retries, exponential backoff, and Flash fallback. Third-party hosts (Together AI, DeepInfra, NVIDIA NIM, OpenRouter) are common backstops. - **Tool-calls leaking into `content`** — intermittent regression where the model emits tool calls as plain text instead of via the `tool_calls` field. Defensive parsing required. - **Multimodal gap** — V4 is text-only. The separate vision variant lags GPT-5.5 native and Gemini 3.1 Pro substantially. Not a fit for image-heavy agents. - **Raw factuality** — SimpleQA-Verified 57.9 vs Gemini 3.1 Pro 75.6. V4 hallucinates more on closed-book factual queries than the closed leaders. - **Tool-use reliability on long-horizon agents** still trails Claude Opus 4.7 (visible in the SWE-bench Pro gap: 55.4 vs 64.3). For ReAct-style agents this is fine; for tool-heavy structured workflows, benchmark before committing. - **402 Insufficient Balance** — not retryable; top up. ## Migrating from Claude or GPT to V4 1. **Swap base URL and model name.** Existing OpenAI-SDK code keeps working — point at `https://api.deepseek.com/v1`, change `model=` to `deepseek-v4-pro` or `deepseek-v4-flash`. 2. **Re-tune system prompts.** V4 follows instructions slightly more literally than Claude. Prompts that rely on Claude's "infer my intent" behavior need to be made explicit. 3. **Decide thinking-mode policy.** Default off for chat-style products (latency); default on for coding agents (quality). Use `thinking: "max"` for the hardest tasks. 4. **Implement `reasoning_content` round-tripping** if you have any multi-turn flows. Skipping this will cost you a day of debugging. 5. **Re-run your eval suite.** Don't trust the published benchmarks — your workload is different. Especially scrutinize tool-heavy and long-horizon scenarios. 6. **Plan for self-hosting** if your token volume justifies it. Crossover with the API is now roughly $150K+/month in spend at the new permanent list price, so most teams should start on the API regardless. The migration is genuinely the kind of work where having an engineer who has done it before saves weeks. [Hiring a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer) typically gets you someone who has already shipped this exact migration. ## What changed in the July 2026 update (V4-Flash) and how V4 compares to the 2026 frontier On July 31, 2026, DeepSeek re-post-trained V4-Flash. The architecture is unchanged from the earlier V4-Flash-Preview — 284B total parameters, ~13B active, a 1M-token context window, and still text-only — but the retraining delivered a significant jump on DeepSeek’s own agentic harness, with Terminal Bench 2.1 landing at 82.7 (alongside Cybergym 76.7, Toolathlon 70.3, and DSBench-FullStack 68.7). Pricing held at $0.14 per million input tokens and $0.28 per million output, with cached input at roughly $0.003 (a 98% discount) that pulls the blended rate down to about $0.06 per million on cache-heavy workloads. Read those vendor numbers as a max-effort ceiling rather than a real-world guarantee, but even discounted they land near-frontier for a model that costs pennies. July 2026 was the month the frontier itself moved, which is what makes the cost gap so stark. Anthropic launched [Claude Opus 5](https://codersera.com/blog/claude-opus-5-complete-guide-2026/) on July 24 at $5 input / $25 output; OpenAI shipped the GPT-5.6 family (Sol at $5 / $30, Terra at $2 / $12, Luna at $0.20 / $1.20 after the July 30 price cut); and Moonshot’s Kimi K3 arrived at $3 / $15 with native vision. Against that board, V4-Flash is roughly 36x cheaper on input and 89x cheaper on output than Opus 5, and about 34x cheaper on output than GPT-5.6 Sol. In a concrete monthly example — 50M input plus 10M output tokens — V4-Flash runs about $9.80 and V4-Pro about $30.45, while the same traffic on Opus 5 is roughly $500 and on GPT-5.6 Sol about $550. The full side-by-side, with every multiplier worked out, is in our [DeepSeek V4-Flash cost vs the frontier breakdown](https://codersera.com/blog/deepseek-v4-flash-cost-vs-frontier-2026/). The honest verdict: for the large majority of production work — code generation, refactoring, retrieval-augmented answering, classification, tool-calling pipelines — V4 gives you near-frontier capability at 2–3% of the cost, and its OpenAI-compatible API makes the switch mostly a base-URL change. Where the flagships still win is the hardest end of the spectrum: the longest-horizon autonomous agents lean toward Opus 5 and GPT-5.6 Sol, and any workload that needs native image, audio, or video is off the table for text-only DeepSeek — that’s where Opus 5, GPT-5.6, and natively-multimodal Kimi K3 earn their premium. Two caveats worth keeping in view: the hosted DeepSeek API runs in China (a data-residency and compliance question for some teams), though the MIT-licensed open weights mean you can self-host and sidestep that entirely. The practical framing for late 2026 is less “which model is best” and more “what can you finally afford to build” — and on that question, V4 is usually the answer for everything short of the hardest agents and multimodal work. ## Which DeepSeek model should you use? - **Coding agents, frontier quality matters most:** V4 Pro ([review](https://codersera.com/blog/deepseek-v4-pro-review-benchmarks-pricing-2026/)) - **Coding agents, cost matters most:** V4 Flash ([deep dive](https://codersera.com/blog/deepseek-v4-flash-deep-dive/)) - **Reasoning-heavy non-coding work:** V4 Pro with `thinking: "max"` — there's almost no reason to use R1 anymore - **Already on V3.2, deciding whether to upgrade:** read the [V3.2 vs V4 changes guide](https://codersera.com/blog/deepseek-v4-vs-deepseek-v32-what-changed-developers-should-use/) — short answer: yes, unless you have V3.2-specific tokenizer hard-coding - **Comparing against the closed-source frontier:** see [V4 vs Qwen, Kimi, MiniMax, GPT, Claude](https://codersera.com/blog/deepseek-v4-alternatives-qwen-kimi-minimax-gpt-claude-compared/) ## Frequently asked questions ### Is DeepSeek V4 free to use? The weights are MIT-licensed and free. The hosted API is paid, but at the new permanent list price (announced May 22, 2026; V4 Pro $0.435/$0.87 per 1M tokens in/out) it's an order of magnitude cheaper than competing frontier APIs. ### Is DeepSeek V4 actually open source? Yes. Both Pro and Flash are MIT-licensed with weights on HuggingFace at `deepseek-ai/DeepSeek-V4-Pro` and `deepseek-ai/DeepSeek-V4-Flash`. Base variants for fine-tuning ship alongside. You can self-host commercially without contacting DeepSeek. ### What's the difference between V4 Pro and V4 Flash? Pro is 1.6T total / 49B active and tops the coding benchmarks. Flash is 284B total / 13B active, slightly behind on quality, dramatically cheaper to self-host. The [Pro vs Flash comparison](https://codersera.com/blog/deepseek-v4-pro-vs-flash/) walks through cost-per-task math. ### Can DeepSeek V4 run on a single GPU? V4 Flash on a single H100 (80GB) at INT4 — yes, but tight. Single H200 (141GB) is the cleanest single-GPU target. V4 Pro requires multi-GPU regardless of quantization. RTX 4090 / 5090: no. ### Can DeepSeek V4 run on a Mac? V4 Flash on M3 Ultra 192GB via MLX (4-bit) — yes, but only for inference and prompt processing is slow (~14 minutes for 8K tokens on the llama.cpp fork). Practical only for development, not production. V4 Pro: no. ### How does V4 compare to Claude Opus 4.7 for coding? SWE-bench Verified: tied (80.6 vs 80.8). LiveCodeBench: V4 ahead (93.5 vs 88.8). Codeforces ELO: V4 ahead (3206 vs not published for Opus). SWE-bench Pro (long-horizon): Opus ahead (64.3 vs 55.4). Pricing at the new permanent list price: V4 Pro is ~34× cheaper on input and ~86× cheaper on output than Opus 4.7. Full breakdown in [V4 vs Claude Opus 4.7](https://codersera.com/blog/deepseek-v4-vs-claude-opus-4-7/). ### How does V4 compare to GPT-5.5? Very close on most benchmarks; V4 Pro slightly ahead on Codeforces, slightly behind on GPQA. V4 is cheaper and open-weight; GPT-5.5 has better tool-use reliability and multimodal. See [V4 vs GPT-5.5 same-week showdown](https://codersera.com/blog/deepseek-v4-vs-gpt-5-5-pro/). ### What is the context window of DeepSeek V4? 1,048,576 tokens (1M) on both variants natively. NIAH retrieval >95% to 900K. Reasoning across the full 1M weakens past ~500K, but for codebase Q&A and long-document tasks this is fine. ### How much does the DeepSeek V4 API cost per million tokens? Permanent list price (effective May 22, 2026): V4 Pro $0.435 input / $0.87 output / $0.003625 cache-hit input. V4 Flash $0.14 input / $0.28 output / $0.0028 cache-hit input. Those rates hold until **16:00 UTC on August 16, 2026**, when both models move to peak/off-peak tiers: V4 Pro to $0.66 / $0.022 / $1.98 off-peak and $1.32 / $0.044 / $3.96 at peak, V4 Flash to $0.22 / $0.007 / $0.66 and $0.44 / $0.014 / $1.32. The previous "75% promo until May 31" wording was retired when DeepSeek made the discount permanent — source: [api-docs.deepseek.com/quick_start/pricing](https://api-docs.deepseek.com/quick_start/pricing). ### Why am I getting a 400 `reasoning_content` error? Multi-turn conversations with thinking mode require you to round-trip `reasoning_content` back as part of the assistant message. Most clients (LiteLLM, OpenCode, OpenClaw, Cursor) don't, which causes the 400. The [API developer guide](https://codersera.com/blog/how-to-use-deepseek-v4-api-developer-guide-2026/) has the full fix. ### How do I use V4 in Cursor or Claude Code? Claude Code: official integration via `https://api.deepseek.com/anthropic`. Cursor: custom model via OpenAI-compat at `https://api.deepseek.com/v1` — be aware of long-session breakage and use the community proxy if needed. ### Does V4 support tool calls and JSON mode? Yes to both. Strict-mode tool calls supported. Critically, V4 supports tool calls *in thinking mode* — R1's biggest limitation is now gone. ### When will `deepseek-chat` and `deepseek-reasoner` retire? July 24, 2026 15:59 UTC. They currently alias to V4 Flash non-thinking and thinking respectively. Migrate to explicit `deepseek-v4-flash` or `deepseek-v4-pro` now. ### Can V4 replace RAG with its 1M context window? For most service-sized codebase Q&A: yes. MRCR 1M of 83.5 and cache-hit pricing make it economical. For multi-document corpora larger than 1M tokens or strict citation requirements, RAG still wins. ### Does V4 support image / multimodal input? Not in the main V4 release. There's a separate vision variant but it lags GPT-5.5 and Gemini 3.1 Pro substantially. For multimodal use cases, look elsewhere. ### How do I fine-tune DeepSeek V4? Use NVIDIA's official NeMo AutoModel recipe for V4 Flash on Blackwell GPUs. Pro fine-tuning needs multi-node — most teams fine-tune Flash and use Pro as a teacher. ## Next steps If you're evaluating V4 for production: 1. Read the [V4 Pro review](https://codersera.com/blog/deepseek-v4-pro-review-benchmarks-pricing-2026/) for the full technical assessment. 2. Run the [local setup guide](https://codersera.com/blog/run-deepseek-v4-flash-locally-full-2026-setup-guide/) on a development H200 to validate quality on your workload. 3. Use the [API developer guide](https://codersera.com/blog/how-to-use-deepseek-v4-api-developer-guide-2026/) to wire V4 into a non-trivial agent and pressure-test the `reasoning_content` handshake. 4. If you're migrating from Claude or GPT and don't have an engineer who has done this before, [hire a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer). Most of our network has shipped at least one production V4 deployment. =============================================================== [2/20] Claude Opus 4.7: The Complete Developer Guide (2026) URL: https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/ Last updated: 2026-08-13 =============================================================== **Quick answer (refreshed May 27, 2026).** Six weeks after Claude Opus 4.7's April 16 launch, Anthropic has **not** shipped Sonnet 4.8 or Claude 5. The names that surfaced in the March 31, 2026 npm source-map leak (codenames **Fennec** for Opus 4.6; **Capybara** / **Mythos** for the post-Opus tier) remain unannounced — no model card, no API id, no benchmarks, and no Anthropic comment. Polymarket gives **September 30, 2026 a 57% lead** for Claude 5; June 30 at 11%. Metaculus median lands late summer. Meanwhile Opus 4.7 has been overtaken on SWE-bench Verified by GPT-5.5 (88.7% vs 87.6%) but still leads **SWE-bench Pro at 64.3%** (the harder, multi-language variant). The real product story since April has been the **Code with Claude 2026** conference wave — Managed Agents in public beta with Outcomes / Dreaming / Multi-Agent Orchestration ($0.08/session-hour + standard rates), webhooks for long-running sessions, self-hosted sandboxes for Cloudflare/Daytona/Modal/Vercel, MCP tunnels, Claude Platform on AWS GA, and a 220k-GPU SpaceX/Colossus compute deal. ## What about Claude Opus 5? (July 2026) The Opus line moved fast after 4.7. Anthropic shipped **Opus 4.8 on May 28, 2026** and then **Claude Opus 5 on July 24, 2026** (API id `claude-opus-5`) — the fourth Claude model in under two months. Opus 5 is a generational jump rather than a point release, and it lands at the **same price as Opus 4.8: $5 per million input tokens and $25 per million output** (a "fast mode" at $10 / $50 trades 2× the price for roughly 2.5× the speed). Anthropic frames it as the everyday workhorse for coding, agentic tasks, and knowledge work — close to frontier intelligence at a fraction of the cost of the larger tier. What makes it a real step up over the 4.x Opus models: a **1M-token context window as both the default and the maximum** (no smaller variant), a per-request **effort toggle (low / medium / high)** to balance compute against reasoning depth, extended thinking on by default, and **self-verification with error recovery** — it checks its own work and iterates until it succeeds, so you spend less time in the back-and-forth loop. On agentic coding it leads outright: **43.3% on Frontier-Bench v0.1** (more than doubling Opus 4.8) and **79.2% on SWE-bench Pro versus Opus 4.8's 69.2%**. It also posts the strongest ARC-AGI-3 novel-reasoning and GDPval knowledge-work scores in Anthropic's lineup, and its automated behavioral audit rates it their most-aligned model to date. This guide remains the reference for the **Opus foundations** — extended thinking, tool use, computer use, MCP, the Messages API, prompt caching, and how the Opus tier is priced and positioned — all of which carry straight into Opus 5. For the newest model's full specs, benchmark tables, pricing tiers (including fast mode), effort-parameter usage, and migration notes, read the dedicated [Claude Opus 5 launch guide](https://codersera.com/blog/claude-opus-5-launch-guide-2026/). ## What's new since April 2026? 1. **Sonnet 4.8 / Claude 5 status: leak only, silence since.** The March 31 npm source-map leak in `@anthropic-ai/claude-code` v2.1.88 referenced Sonnet 4.8 plus codenames Fennec, Capybara, and Mythos. Anthropic has said nothing publicly in the eight weeks since. Treat any "Sonnet 4.8 imminent" prediction as speculative — bake the pillar around Opus 4.7 + Managed Agents for the next 2-4 months. 2. **Code with Claude 2026 conference wave.** Three city events (San Francisco May 6, London May 19, Tokyo June 10) shipped: **Managed Agents** (public beta) with Multiagent Orchestration, Outcomes (define success criteria, agent self-iterates), Dreaming (recall and build on prior sessions). Pricing: standard per-MTok rates + $0.08/session-hour billed to the millisecond. Direct API only — not on Bedrock or Vertex. **Webhooks** for Managed Agents went public beta on May 7 — signed, retry semantics, stable event schema. Replaces polling for long-running sessions. 3. **Self-hosted sandboxes + MCP tunnels.** Pre-built environment workers for Cloudflare, Daytona, Modal, Vercel; tool execution runs on customer infrastructure while the agent loop stays on Anthropic. Public beta sandboxes; research-preview MCP tunnels. 4. **Claude Platform on AWS GA (May 2026)** — native Anthropic experience inside an AWS account, separate from Bedrock. Pairs with the SpaceX/Colossus capacity allocation — all 220k+ GPUs / 300+ MW of Colossus 1 capacity now allocated to Claude. Q1 2026 revenue grew 80× annualised vs the 10× planned. 5. **Claude Code 2.1.x updates.** `/simplify` renamed to `/code-review` with effort levels; `--comment` posts findings as inline GitHub PR comments. **Pinned background sessions** (Ctrl+T in agent view) stay alive when idle, restart in place across CC updates, shed only under memory pressure after non-pinned sessions. `/resume` now works for background sessions. `pluginSuggestionMarketplaces` managed setting for admin-allowlisted org marketplaces. 6. **Computer Use: GA in Cowork, beta on the API.** Claude Cowork reached GA with Computer Use for Pro/Max users in April. API still beta, still requires the beta header. Opus 4.7 scores **77.9% on OSWorld** (Opus 4.6: 72.7%) — meaningful for real-screen agent tasks. 7. **Opus 4.7 pricing — unchanged but watch the tokenizer.** Still $5 / $25 per 1M tokens (67% below the Opus 4/4.1 era). However the new tokenizer can produce up to 35% more tokens for the same input text — real bills can rise even though the rate card didn't. Prompt-cache reads $0.50/MTok; batch processing 50% off. 8. **SWE-bench leaderboard moved.** GPT-5.5 (88.7%, April 23) overtook Opus 4.7 (87.6%) on SWE-bench Verified. Opus 4.7 still leads SWE-bench Pro at 64.3% (the harder, multi-language version). Claude Mythos Preview (Anthropic's restricted preview tier — likely Capybara) sits at 93.9% on Verified — visible on leaderboards but not publicly accessible. 9. **Adaptive Thinking gotcha.** Extended-thinking `budget_tokens` is removed in Opus 4.7 — passing it returns 400. Adaptive thinking is the only thinking-on mode; the model self-allocates reasoning per request. Task budgets now cover the full agentic loop (thinking + tool calls + final output) with a running countdown the model can see. High-res image input lifted to 2576 px / 3.75 MP (was 1568 px / 1.15 MP). For the broader landscape see our [GPT-5.5 guide](https://codersera.com/blog/gpt-5-5-complete-guide-2026/), [Gemini 3.5 guide](https://codersera.com/blog/gemini-3-5-complete-guide-2026/), [AI coding agents pillar](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/), and the [open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/). The rest of this guide covers Opus 4.7's capabilities, extended thinking, Computer Use, pricing, and deployment patterns in depth. *Last updated: May 1, 2026* Anthropic shipped Claude Opus 4.7 on April 16, 2026, just over two months after Opus 4.6. On paper it is an incremental release. In practice it is the first model where you can hand off a multi-hour engineering task and reasonably expect it to come back with a working pull request. It also ships with a new tokenizer, a reworked thinking API, and a few regressions that will quietly raise your bill if you copy your old prompts forward. This guide is for engineering leaders, founders, and developers deciding where Opus 4.7 fits in a 2026 stack alongside Sonnet 4.6, Haiku 4.5, GPT-5.5, and DeepSeek V4 Pro. We focus on what changed, what the API actually costs, where it beats and loses to its peers, and when it is the wrong tool for the job. ## TL;DR - **What it is:** Anthropic's flagship reasoning and coding model, released April 16, 2026. Same $5 / $25 per million tokens (input / output) as Opus 4.6, but a new tokenizer that produces roughly 1.0–1.35x more tokens per request. - **Why it matters:** 87.6% on SWE-bench Verified and 64.3% on SWE-bench Pro — a clear lead over GPT-5.5 (58.6%) and DeepSeek V4 Pro (55.4%) on real-world software engineering tasks. - **Where it loses:** GPT-5.5 still leads the Artificial Analysis Intelligence Index (60 vs 57) and on terminal/agent breadth. DeepSeek V4 Pro is roughly 29x cheaper at $0.87 per million output tokens (the $1.74 / $3.48 list rates were superseded on 2026-05-22; DeepSeek raises prices again on August 16, 2026). - **What is new:** 1M-token context at standard pricing, adaptive thinking only (manual budgets removed), 3.75 MP vision, MCP-Atlas score of 77.3%, task budgets in beta, and a new `xhigh` effort level in Claude Code. - **What broke:** Web research and source-attribution accuracy regressed. Long-form prose got more mechanical. Code comments dropped from ~8% to ~4% of output. Manual `thinking.budget_tokens` calls now error. - **Bottom line:** Default to Sonnet 4.6 for 80% of work. Reach for Opus 4.7 when the task is hard enough that one Opus run beats five Sonnet retries. Use Haiku 4.5 for high-volume routing, classification, and extraction. ## What changed from Opus 4.5 and 4.6 Anthropic shipped three Opus-class models in five months: 4.5 in December 2025, 4.6 on February 5, 2026, and 4.7 on April 16, 2026. The cadence is fast enough that "should I upgrade" is a real question, not a reflex. The headline gains over Opus 4.6: - **Coding:** SWE-bench Verified jumps from 80.8% (4.6) to 87.6% (4.7). CursorBench moves from 58% to 70%. - **Vision:** 4.7 accepts images up to 2,576 px on the long edge (~3.75 megapixels), more than 3x the resolution of 4.6. XBOW visual-acuity scores went from 54.5% to 98.5%. - **Agentic execution:** 14% better on multi-step workflows at fewer tokens, with a third of the tool errors of 4.6. First Claude model to pass implicit-need tests and to recover gracefully from tool failures that used to halt the agent. - **Instruction following:** More literal. 4.6 was loose and would silently skip steps; 4.7 does what you asked, including when what you asked was wrong. Prompts tuned for 4.6 will need re-reading. And the things that quietly broke: - **New tokenizer.** The same English text produces 1.0–1.35x more tokens than under Opus 4.6's tokenizer. Per-token rates are unchanged; per-request bills can rise meaningfully. Re-cost your workloads before migrating. - **Manual thinking budgets removed.** `thinking: {type: "enabled", budget_tokens: N}` is no longer accepted. 4.7 only supports adaptive thinking and decides per-step how much to think. - **Thinking content omitted by default.** Thinking blocks still stream, but the `thinking` field is empty unless you opt in. - **Web research regressed.** Source attribution, contradiction detection, and citation specificity are all worse than 4.6 in head-to-head testing. If you run a research agent that grounds claims in cited sources, validate before flipping the model. For a deeper side-by-side with the leading open-weights challenger, see our [DeepSeek V4 vs Claude Opus 4.7 comparison](https://codersera.com/blog/deepseek-v4-vs-claude-opus-4-7/). ## Architecture: extended thinking, tool use, computer use, MCP Opus 4.7 is built around four primitives that, together, define what "agentic coding" means in 2026. ### Adaptive thinking Adaptive thinking is now the only thinking mode on Opus 4.7. The model decides per turn whether to run a hidden chain-of-thought and how long it should be. For trivial questions it skips thinking entirely and answers in one round trip. For a SWE-bench-grade bug fix it can think for tens of thousands of tokens before emitting a single character of code. The trade-off: you lose the deterministic ceiling that `budget_tokens` gave you in 4.6. To bound spend, use the new *task budgets* beta, which sets a hard token ceiling on an agentic loop and lets the model see a running countdown so it can finish gracefully instead of cutting off mid-task or surprising you with the bill. ### Tool use and MCP Tool use is unchanged at the protocol level: you declare tools in the request, Claude emits structured `tool_use` blocks, you respond with `tool_result`. The Model Context Protocol (MCP) is now the de-facto standard for connecting Claude to filesystems, databases, browsers, and internal services without wiring each one into your prompt. The behavioral upgrade matters more than the protocol. Opus 4.7 scores 77.3% on MCP-Atlas, a benchmark for scaled multi-tool agentic tasks, and it keeps executing through tool failures that would have halted Opus 4.6. That is the difference between an agent you can let run for an hour and one you have to babysit. ### Computer use The computer-use API (beta header `computer-use-2025-11-24`) lets Claude take screenshots, move a cursor, click, type, and scroll in a real desktop environment. With 4.7's higher-resolution vision, it can finally read dense web UIs, full-screen IDEs, and design tools without losing detail. It is still beta, still slow, and still best run in a sandboxed VM. ### Extended-context coding The 1M-token context window is now standard pricing on Opus 4.7. That is enough to load a mid-size monorepo, a few hundred pages of design docs, or a long Slack thread alongside the actual prompt. Long-context retrieval did not improve uniformly — precise ordinal recall over hundreds of thousands of tokens is still slightly stronger on Opus 4.6. For most agentic coding work, 4.7 wins anyway. ## API basics: Messages, caching, batch, citations, files You call Opus 4.7 with the model id `claude-opus-4-7` against the standard Messages API. The platform features that matter for cost and latency: - **System prompts:** Standard. Place your tool definitions and reusable context in the system block so they are eligible for caching. - **Prompt caching:** Mark a content block with `cache_control: {type: "ephemeral"}` and Anthropic stores the prefix. Cache writes cost 1.25x input ($6.25/M for 5-minute TTL, $10/M for 1-hour TTL). Cache reads are 10% of input ($0.50/M). Minimum cache size on Opus 4.7 is 4,096 tokens. - **Batch API:** Submit asynchronous jobs and get a 50% discount on both input and output. Effective Opus 4.7 batch rate is $2.50 / $12.50 per million. Combine with caching and your effective input rate drops to roughly $0.25 per million. - **Citations:** Pass documents as content blocks with `citations.enabled = true` and Claude grounds responses in the specific sentences it used. Citations work alongside caching — the source documents cache, the per-response citation blocks do not. - **Files API:** Upload a file once, reference it by id from any future Messages request. Useful for repeated PDF, image, or codebase inputs. For migration patterns from earlier Claude versions, see [how to use Claude 4 and Sonnet with Cursor and Windsurf](https://codersera.com/blog/how-to-use-claude-4-and-sonnet-with-cursor-and-windsurf/), which covers the IDE-side wiring you will reuse with 4.7. ## Benchmarks: what the numbers actually say Benchmarks are useful for narrowing your shortlist, not for picking a winner. The table below is the current snapshot for Opus 4.7 against the two models it gets compared to most often: GPT-5.5 (high effort) and DeepSeek V4 Pro (max effort). | Benchmark | Opus 4.7 | GPT-5.5 | DeepSeek V4 Pro | What it measures | | --- | --- | --- | --- | --- | | SWE-bench Verified | **87.6%** | 79.2% | ~76% | Real GitHub issue fixes | | SWE-bench Pro | **64.3%** | 58.6% | 55.4% | Harder multi-language SWE tasks | | LiveCodeBench | 78.5% | ~80% | **~82%** | Competitive programming | | Terminal-Bench 2.0 | ~75% | **82.7%** | 67.9% | Shell agent tasks | | MCP-Atlas | **77.3%** | ~74% | ~65% | Multi-tool agentic workflows | | GPQA Diamond | 94.2% | 93.6% | ~88% | Graduate-level science | | MMLU-Pro | 89.9% | ~91% | ~87% | Broad knowledge | | IFEval | 91.2% | ~92% | ~89% | Instruction following | | HLE (Humanity's Last Exam) | 54.7 | 52.2 | 37.7 | Frontier reasoning | | AA Intelligence Index | 57 | **60** | ~50 | Composite | The honest read: Opus 4.7 is the best model on the market for the specific shape of work that is "fix a real bug in a real repo" or "drive a multi-step tool-using agent." GPT-5.5 still has the breadth lead and wins on terminal-style tasks. DeepSeek V4 Pro wins on competitive programming and on cost. For more on the open-source side, see our [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). For older head-to-heads that are still useful for context, see [Llama 4 vs Claude 3.7 Sonnet](https://codersera.com/blog/llama-4-vs-anthropic-claude-3-7-sonnet/) and the [DeepSeek V3.1 Terminus vs GPT-5 vs Claude 4.1 comparison](https://codersera.com/blog/deepseek-v31-terminus-vs-chatgpt-5-vs-claude-41-2025-comparison-guide/). ## Pricing across the Anthropic lineup Per-token pricing on Opus 4.7 is unchanged from 4.6. The catch is the new tokenizer: the same input text now produces up to 35% more tokens, so your per-request cost can drift up even on identical workloads. | Model | Input ($/M) | Output ($/M) | Cache write 5m ($/M) | Cache read ($/M) | Batch (50% off) | Context | | --- | --- | --- | --- | --- | --- | --- | | Claude Opus 4.7 | $5.00 | $25.00 | $6.25 | $0.50 | $2.50 / $12.50 | 1M | | Claude Sonnet 4.6 | $3.00 | $15.00 | $3.75 | $0.30 | $1.50 / $7.50 | 1M | | Claude Haiku 4.5 | $1.00 | $5.00 | $1.25 | $0.10 | $0.50 / $2.50 | 200K | A 1-hour cache TTL is also available at 2x the input rate ($10/M on Opus 4.7) and pays for itself once cache reads exceed about eight per stored prefix. ### Real-world cost: a coding-agent workload Consider an autonomous coding agent that fixes 100 medium-complexity bugs per day. A typical run looks like 50,000 tokens of cached context (codebase, conventions, system prompt), 5,000 tokens of fresh input per task, and 8,000 tokens of generated output (thinking + final code). - **First-task cost:** 50K cache write @ $6.25/M = $0.31, plus 5K input @ $5/M = $0.025, plus 8K output @ $25/M = $0.20. Total: **$0.54**. - **Subsequent task cost (cache warm):** 50K cache read @ $0.50/M = $0.025, plus 5K input @ $5/M = $0.025, plus 8K output @ $25/M = $0.20. Total: **$0.25**. - **Daily total (100 tasks):** $0.54 + 99 × $0.25 = **~$25/day**, or roughly $750/month. - **Same workload on DeepSeek V4 Pro:** approximately **$29/month** at its current $0.435 / $0.003625 / $0.87 rates — about **26x cheaper**, with a measurable but small quality drop on hard fixes. (That earlier $110 figure was computed against DeepSeek's retired $1.74 / $3.48 list price. From 16:00 UTC on August 16, 2026 the same workload runs roughly $62/month off-peak and $123/month at peak, still 6–12x cheaper than Opus 4.7 — see [the DeepSeek price change breakdown](https://codersera.com/blog/deepseek-v4-price-change-august-2026/).) - **Same workload on Sonnet 4.6:** roughly **$450/month**, with a meaningful quality drop on hard fixes that often shows up as failed tests and retry loops. The economic question is rarely "is Opus 4.7 worth $25 more per output million than Sonnet 4.6." It is "does one Opus run beat 2-5 Sonnet retries on this task." For senior-grade engineering work the answer is usually yes. For routine refactors and templated CRUD, it is usually no. ## Claude Code and agent capabilities Opus 4.7 is the default model in Claude Code as of mid-April 2026, with a new `xhigh` effort level sitting between `high` and `max`. `xhigh` is now the default for Opus 4.7 in Claude Code — Anthropic's own bet that the extra latency is worth it for the quality jump on hard problems. What you actually feel using Claude Code with 4.7: - Long-running tasks that used to die at the 30-minute mark now run for hours and recover from individual tool failures. - Context carries across sessions more reliably — you can stop, come back the next day, and pick up without re-priming. - Ambiguous instructions get clarifying questions less often; the model commits to a path and executes. This is good when you are right and bad when you are not. - Security tuning is over-eager. Claude Code 4.7 has been seen flagging static HTML/CSS as potential malware and refusing edits. This is a tuning regression that will likely be patched, but it is worth knowing about before you put it in front of a junior dev. If you want to run an open-source equivalent for evaluation, see our guide to [running open-source Claude Code OSS](https://codersera.com/blog/how-to-run-open-source-claude-code-oss-developer-guide-2026/). ## When to use Opus 4.7 vs Sonnet 4.6 vs Haiku 4.5 The most expensive mistake in 2026 is sending all your traffic to Opus. The second most expensive is sending all of it to Haiku. A three-tier router pays for itself within weeks at any non-trivial volume. | Use case | Recommended model | Why | | --- | --- | --- | | Hard SWE-bench-grade bugs, architectural design, deep code review | Opus 4.7 (xhigh) | Quality lead is decisive; one good run beats 3-5 retries. | | Day-to-day feature work, PR review, content generation, RAG answers | Sonnet 4.6 | Best capability-per-dollar; handles 80% of production work. | | Routing, classification, extraction, summarisation, chat first-line | Haiku 4.5 | 3x cheaper than Sonnet, fast enough for real-time. | | Long-document precise ordinal retrieval | Opus 4.6 (still) | 4.7 regressed slightly on this narrow case. | | Web research with strict citation accuracy | Opus 4.6 or GPT-5.5 | 4.7's source attribution regressed. | | Bulk inference where cost dominates quality | DeepSeek V4 Pro | ~7x cheaper; trails on hard tasks but close enough for many. | ## Known limitations Anthropic's launch post for Opus 4.7 is, by Anthropic standards, unusually candid about what the model is not. The honest picture from third-party reviews: 1. **Token inflation.** The new tokenizer increases token counts by 12–35% on typical inputs. Your per-request bills will rise even at unchanged per-token rates. 2. **Web research and citation accuracy regressed.** The model is more likely to attribute a claim to the wrong source or paper over conflicting sources. If you ship a research agent, do not migrate without an A/B. 3. **Code comments dropped.** Comment density went from 8.2% of output on 4.6 to 3.8% on 4.7. Code is more compact and slightly harder to maintain. Some static-analysis tools (Sonar) report a small increase in blocker/critical findings vs 4.6. 4. **Long-form prose got more mechanical.** Opus 4.7 reaches for bullets and headings where 4.6 held a flowing narrative. Marketing and editorial teams may prefer to keep 4.6 in the loop for first drafts. 5. **Competition math.** Around 70% on USAMO 2026 — well behind GPT-5.4 and 5.5 on that specific benchmark. Opus 4.7 is not the model to point at olympiad-grade math. 6. **API surface changes.** Manual thinking budgets are gone. Thinking content is hidden by default. Beta headers shifted. Migration is not a model-id swap; re-read your client code. 7. **Overzealous safety filtering in Claude Code.** Benign code occasionally flagged. Expect this to be patched but plan around it for now. 8. **It still trails GPT-5.5 on the AA Intelligence Index** (57 vs 60). Anthropic itself acknowledged Opus 4.7 trails the unreleased Mythos model. The frontier is moving every quarter. ## Comparing Opus 4.7 to GPT-5.5 and DeepSeek V4 Pro The 2026 frontier is a three-way race rather than a single leader. **vs GPT-5.5.** GPT-5.5 wins on overall intelligence index, on terminal/agent breadth (Terminal-Bench 2.0 at 82.7% vs Opus 4.7's ~75%), and on cost-per-quality at the top end. Opus 4.7 wins decisively on SWE-bench Verified and SWE-bench Pro, on MCP-Atlas, and on long-running coding agents that need to recover from tool failures. Pick GPT-5.5 if your agent surfaces are heterogeneous and your bottleneck is "can it operate this CLI." Pick Opus 4.7 if your bottleneck is "can it ship this PR." **vs DeepSeek V4 Pro.** DeepSeek V4 Pro is roughly 29x cheaper per output token ($0.87/M vs $25/M — and 6–13x cheaper after DeepSeek's August 16, 2026 price rise), open-weights, and within striking distance on most benchmarks (within 5–10 points on SWE-bench Pro and GPQA Diamond). It actually leads on competitive programming. The gap shows on long-horizon agentic work and on hard, novel bug fixes — the exact areas where Opus 4.7 invested. For high-volume inference, internal tools, and cost-sensitive products, DeepSeek V4 Pro is the rational default. For mission-critical engineering work, Opus 4.7 still earns its premium. ## FAQ ### What is the model id for Claude Opus 4.7 in the API? `claude-opus-4-7`. Use it as the `model` parameter in the Messages API. ### Did pricing change from Opus 4.6 to 4.7? No. Per-token pricing is identical at $5 input / $25 output per million. Effective per-request cost rises because the new tokenizer produces 1.0–1.35x more tokens for the same English text. ### Does Opus 4.7 support a 1M-token context window? Yes, at standard pricing. Output is capped at 128K tokens. ### Is manual `thinking.budget_tokens` still supported? No. Opus 4.7 supports adaptive thinking only; the manual budget parameter now errors. Use the new task-budgets beta to bound spend on agentic loops. ### How much does prompt caching save? Cache reads cost 10% of the standard input rate ($0.50 per million tokens on Opus 4.7). Cache writes cost 1.25x input for a 5-minute TTL. The break-even point is roughly two cache reads per write. ### How much does the Batch API save? 50% off both input and output, bringing effective rates to $2.50 / $12.50 per million on Opus 4.7. Batches can take up to 24 hours and are best paired with the 1-hour cache TTL for shared context. ### Should I migrate from Opus 4.6 to 4.7 today? Yes for coding, agentic workflows, and vision tasks. Stay on 4.6 if your workload is web research with strict citation accuracy, long-form creative prose, or precise ordinal retrieval over hundreds of thousands of tokens. ### Is Opus 4.7 better than GPT-5.5? For software engineering, yes — it leads on SWE-bench Verified (87.6% vs 79.2%) and SWE-bench Pro (64.3% vs 58.6%). For broad intelligence and terminal/agent breadth, GPT-5.5 still leads. ### Is Opus 4.7 better than DeepSeek V4 Pro? On hard, novel coding tasks and long-horizon agents, yes. On cost-per-quality, no — DeepSeek V4 Pro is roughly 7x cheaper per output token and competitive on most benchmarks. Use both: Opus 4.7 for the hardest work, DeepSeek V4 Pro for bulk. ### Does Opus 4.7 work with Cursor and Windsurf? Yes. Both Cursor and Windsurf added Opus 4.7 to their model picker within days of launch. See our guide on [using Claude 4 and Sonnet with Cursor and Windsurf](https://codersera.com/blog/how-to-use-claude-4-and-sonnet-with-cursor-and-windsurf/) for setup patterns that carry forward. ### Does Opus 4.7 support computer use? Yes, via the `computer-use-2025-11-24` beta header. The 3.75 MP vision upgrade makes it noticeably better at reading dense web UIs and IDEs than earlier versions. ### Where is Opus 4.7 available besides the Anthropic API? Amazon Bedrock, Google Cloud Vertex AI, Microsoft Foundry, Snowflake Cortex AI, GitHub Copilot Pro+, and Claude Code. Pricing parity varies by platform; the Anthropic-direct API is usually the cheapest. ### What is task budgets and when should I use it? A beta primitive that sets a hard token ceiling on an agentic loop and exposes a running countdown to the model so it prioritises and finishes gracefully. Use it whenever you let Opus run unsupervised for more than a few minutes. ### Will Opus 4.7 replace human engineers? No. It will replace engineers who do not use it. The bottleneck for shipping software is still architecture, code review, judgement on tradeoffs, and accountability for production. Opus 4.7 is a force multiplier on a senior engineer; it is not a substitute for one. ## Next steps If you are deciding where Opus 4.7 fits in your stack, the cheapest experiment is also the most informative: pick one workflow, route it to Opus 4.7 for a week, and measure. If the workflow is "ship more software, faster, with fewer regressions," you also need engineers who can wire it up properly — prompt caching, MCP, task budgets, evals, the lot. [**Hire a Codersera-vetted Python or AI engineer**](https://codersera.com/blog/hire/hire-python-developer) to integrate Opus 4.7 into your codebase, build the routing layer that sends the right task to the right model, and stand up the evals that tell you whether it is actually working. Vetted, remote-ready, and available in days — not months. =============================================================== [3/20] GPT-5.5: The Complete Developer Guide (2026) URL: https://codersera.com/blog/gpt-5-5-complete-guide-2026/ Last updated: 2026-08-13 =============================================================== **Quick answer (refreshed May 27, 2026).** GPT-5.5 now ships in four flavours, not two: the original **GPT-5.5** base model (April 23, 2026 launch), **GPT-5.5 Pro** (highest-accuracy / parallel-reasoning tier), and as of May 5, **GPT-5.5 Instant** (the new free-tier ChatGPT default that replaced GPT-5.3 Instant). The reasoning line — **o3 and o4-mini** — took an 80% price cut in March 2026 and now beats GPT-5.5 on cost-per-correct-answer for math and PhD-level science tasks. **GPT-6 has not shipped**; Polymarket prices it at ~85% by end of 2026, only 49% by September, and 15% by June. OpenAI's bigger 2026 product moves around the model line: **ChatGPT Atlas** (the new OpenAI-built browser, absorbed the Operator product), **Workspace Agents** (GA April 22 for Business/Edu), **Codex on GPT-5.5** (rolled forward in April, ~4M weekly active devs, now on Windows + mobile). For everyday chat, Instant is the right default. For coding agents, GPT-5.5 base. For accuracy-critical legal/medical/scientific work, GPT-5.5 Pro. For tight-budget reasoning, o4-mini. 🆕 **Newer generation:** OpenAI has since previewed [GPT-5.6](https://codersera.com/blog/gpt-5-6-release-date-whats-new-2026/) — a three-tier family (**Sol**, **Terra**, **Luna**) with Terra positioned as GPT-5.5 quality at roughly half the cost. This guide still covers GPT-5.5, which remains widely available; for the newest models see the [GPT-5.6 Sol, Terra & Luna guide](https://codersera.com/blog/gpt-5-6-sol-terra-luna/). ## What's new in GPT-5.5 between April and May 27, 2026? Five things that move the picture since the original guide shipped: 1. **GPT-5.5 Instant (May 5, 2026)** replaces GPT-5.3 Instant as the free-tier ChatGPT default. AIME 2025 jumped from 65.4% to 81.2% versus 5.3 Instant; MMMU-Pro rose from 69.2% to 76%. Lower hallucination in law / medicine / finance. API pricing matches the standard GPT-5.5 tier; in ChatGPT it's free. The model picker is now Instant / Thinking / Pro — the same shape as the Gemini picker. 2. **o3 and o4-mini are dramatically cheaper.** An 80% price cut in March 2026 put o3 at $2 in / $8 out and o4-mini at $1.10 / $4.40 per 1M tokens — roughly 5–6× cheaper than GPT-5.5 on output. o3 hits 87.7% GPQA Diamond, 88.9% AIME 2025, 69.1% SWE-bench Verified. For math contest problems or step-by-step reasoning at budget, the reasoning line is still the right call. 3. **ChatGPT Atlas** launched as OpenAI's first browser (macOS first, Free + Plus + Pro + Go, with Business/Enterprise beta). It absorbed the Operator product — `operator.chatgpt.com` is being deprecated, with Operator's virtual-browser functionality folded into ChatGPT Agent. 4. **Workspace Agents** went GA on April 22 for Business and Edu — teams build Codex-powered shared agents that run inside ChatGPT or Slack. **Codex** itself moved from GPT-5.3-Codex to GPT-5.5 in April, expanded to Windows in March and mobile in mid-May. About 4M weekly active developers per OpenAI's own figures. 5. **GPT-6 ETA** remains soft. Sam Altman confirmed in March that pre-training for the next frontier model completed at the Stargate Abilene data center. Polymarket gives ~85% by end of 2026, 49% by Sept 30, 15% by June 30. The fact that "Spud" shipped as 5.5 rather than 6 was itself a signal — expect another 5.x release before a generation jump. ## GPT-5.5 family at a glance (late May 2026) | Model | Input $/1M | Output $/1M | Context | Best for | | --- | --- | --- | --- | --- | | GPT-5.5 (standard) | $5.00 | $30.00 | 1M | Coding, agentic flows, long-context | | GPT-5.5 Pro | $30.00 | $180.00 | 1M | Legal, finance, science — accuracy-critical | | GPT-5.5 Instant | $5.00 API · free in ChatGPT | $30.00 API | 1M | Default ChatGPT model, low-latency Q&A | | o3 | $2.00 | $8.00 | 200K | Math / science reasoning on a budget | | o4-mini | $1.10 | $4.40 | 200K | Cheapest reasoning; near-o3 on AIME | For the rest of the GPT-5.5 surface — feature list, multimodal, function calling, structured outputs, the Responses API — read on. The comparison vs Claude Opus 4.7 and Gemini 3.5 sits in our [Claude Opus 4.7 guide](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/) and [Gemini 3.5 guide](https://codersera.com/blog/gemini-3-5-complete-guide-2026/). *Last updated: May 1, 2026.* OpenAI shipped GPT-5.5 on April 23, 2026, with API access following on April 24. It is, by every public benchmark we have, the strongest agentic coding model on the market today, and the first OpenAI model that genuinely feels like it can sit inside a software engineering pipeline and carry multi-step work end to end without constant supervision. It is also expensive, verbose at high reasoning effort, and not the right default for every job. This guide walks through what the model actually does, what the variants and reasoning levels mean for cost and latency, how it compares to **Claude Opus 4.7** and **DeepSeek V4 Pro**, and where each one wins. ## TL;DR - **Two API variants:** `gpt-5.5` and `gpt-5.5-pro`. Five reasoning levels: `none`, `low`, `medium` (default), `high`, `xhigh`. - **Pricing:** $5 / $30 per 1M input/output tokens for `gpt-5.5`, with a 90% cached-input discount ($0.50). `gpt-5.5-pro` is $30 / $180 per 1M and does *not* offer a cached-input discount. Batch API is 50% off. - **Context:** 1,000,000 tokens. - **Headline benchmarks:** 82.7% on Terminal-Bench 2.0 (state of the art), 82.6% on SWE-bench Verified, 58.6% on SWE-bench Pro, Intelligence Index 60 on Artificial Analysis (xhigh). - **Where it wins:** agentic tool use, long-running terminal/CLI workflows, token efficiency vs. Claude Opus 4.7 (~72% fewer output tokens on identical coding tasks). - **Where it loses:** Claude Opus 4.7 still leads on SWE-bench Pro (64.3% vs 58.6%) and broad architectural reasoning. DeepSeek V4 Pro is roughly 1/11th the input price and 1/34th the output price for ~85% of the capability on most non-frontier work. - **Recommended API surface:** Responses API, not Chat Completions. Web search and computer use are first-class tools. ## What GPT-5.5 actually is GPT-5.5 (codename "Spud") is OpenAI's flagship general-purpose reasoning model, succeeding GPT-5.4 and the GPT-5.3-Codex variant. OpenAI positions it as their strongest agentic coding model, but the model card and the Artificial Analysis evaluations both emphasize that the gains are broader than coding: it is also better at knowledge work, document and spreadsheet manipulation, computer use, and operating across tools without losing the thread of the task. The most useful way to think about it: GPT-5.5 is the first OpenAI model where the "agent loop" feels like a first-class product surface rather than a wrapper over a chat model. That shows up in the API design (the Responses API has displaced Chat Completions for new projects), in tooling (built-in web search, computer use, file search, code interpreter, and remote MCP all live in the same primitive), and in the way Codex CLI and the Agent SDK route through GPT-5.5 by default. If you're new to the GPT-5 family, our [DeepSeek V3.1 Terminus vs. ChatGPT-5 vs. Claude 4.1 comparison](https://codersera.com/blog/deepseek-v31-terminus-vs-chatgpt-5-vs-claude-41-2025-comparison-guide/) covers the GPT-5 baseline; this guide assumes that lineage and focuses on what changes at 5.5. ## Variants, reasoning levels, and pricing OpenAI ships GPT-5.5 in two API SKUs and three ChatGPT surfaces. The reasoning-effort knob is the single most consequential dial in your API request — it changes both quality and the number of "reasoning tokens" you're billed for. ### API SKUs and pricing per 1M tokens | Model | Input | Cached input | Output | Batch (50% off) | Context | Best for | | --- | --- | --- | --- | --- | --- | --- | | `gpt-5.5` | $5.00 | $0.50 (90% off) | $30.00 | $2.50 in / $15.00 out | 1,000,000 | Default agentic coding, knowledge work, tool use | | `gpt-5.5-pro` | $30.00 | not offered | $180.00 | $15.00 in / $90.00 out | 1,000,000 | Frontier reasoning, research-grade tasks, evals where every percentage point matters | ### Reasoning levels Both SKUs accept `reasoning.effort` with five values: `none`, `low`, `medium` (default), `high`, and `xhigh`. Higher effort burns more reasoning tokens (billed as output) and pushes time-to-first-token up sharply — Artificial Analysis measured TTFT around 115 seconds for `gpt-5.5` at `xhigh`, versus a couple of seconds at `medium`. `medium` is the recommended starting point. Use `xhigh` only for jobs where a few extra dollars and a couple of minutes of thinking are worth it; in practice that's competition math, multi-file refactors with subtle invariants, or research-grade analysis. Note that as of GPT-5.4, tool calling is no longer supported in Chat Completions when `reasoning.effort` is `none` — another reason to default to the Responses API. ### Prompt caching mechanics Caching is automatic. Any prompt of 1,024 tokens or more becomes a candidate, with cache lookups happening on a per-server basis. Place your stable preamble (system prompt, tool definitions, retrieved-doc context) at the front; put per-request variables at the end. For `gpt-5.5`, cached entries persist for 24 hours by default — much longer than the 5-10 minute window earlier GPT-5 models used. `in_memory` caching is no longer offered for 5.5+. The cached-input price drops to $0.50 per 1M, a 90% discount, but again: this discount does not apply to `gpt-5.5-pro`. ## Benchmarks: GPT-5.5 vs. Claude Opus 4.7 vs. DeepSeek V4 Pro The benchmarks worth caring about for engineering buyers are the ones tied to real workflows: SWE-bench Pro and Verified (does the model actually fix GitHub issues?), Terminal-Bench 2.0 (can it drive a CLI through a multi-step task?), LiveCodeBench (held-out competitive programming, harder to memorize), GPQA Diamond (PhD-level reasoning), AIME 2025 (Olympiad math), and MMLU-Pro (a contamination-resistant successor to MMLU). Here is the snapshot at launch. | Benchmark | GPT-5.5 (xhigh) | Claude Opus 4.7 | DeepSeek V4 Pro (Max) | Notes | | --- | --- | --- | --- | --- | | Artificial Analysis Intelligence Index | 60 | ~57 | 52 | GPT-5.5 leads the composite at launch. | | SWE-bench Verified | 82.6% | ~80% | ~78% | Some specialist agents (e.g. Mythos) score higher with scaffolding. | | SWE-bench Pro | 58.6% | 64.3% | 55.4% | Opus 4.7 still leads end-to-end repo work. | | Terminal-Bench 2.0 | 82.7% | 69.4% | 67.9% | GPT-5.5 is well ahead on CLI/agent loops. | | LiveCodeBench | ~88% | ~83% | ~80% | Pass@1 on held-out contest problems. | | GPQA Diamond | ~89% | ~87% | ~82% | PhD-level science MCQ. | | AIME 2025 | ~96% | ~92% | ~93% | Olympiad math, short-answer. | | MMLU-Pro | ~87% | ~86% | ~83% | Contamination-resistant MMLU successor. | | BrowseComp | 84.4% | ~78% | 83.4% | DeepSeek closes a lot of the gap on web research. | The honest one-line summary: GPT-5.5 leads the agentic and tool-use evals, Claude Opus 4.7 still leads codebase-resolution evals like SWE-bench Pro and CursorBench, and DeepSeek V4 Pro is closer to both than its price tag suggests, especially on browsing, terminal work, and MCP Atlas. For a deeper benchmark-by-benchmark dive on the cheaper alternative see our [DeepSeek V4 vs. GPT-5.5 Pro head-to-head](https://codersera.com/blog/deepseek-v4-vs-gpt-5-5-pro/) and the [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). ### Token efficiency, not just accuracy One number that almost never makes the marketing chart but shows up immediately in your bill: on the same coding task, with the same prompt and the same goal, GPT-5.5 produces roughly **72% fewer output tokens** than Claude Opus 4.7. That partially neutralizes Opus's per-token price advantage and is one reason routing layers (Cursor, Codex, Cline) increasingly pick GPT-5.5 by default and only escalate to Opus on jobs where the extra architectural reasoning is worth the verbosity premium. ## What's new vs. GPT-5 and GPT-5 Codex GPT-5.5 is more an evolution than a re-architecture, but several capabilities cross the line from "preview" to "production": - **Agentic coding takes the front seat.** GPT-5.3-Codex was the specialist; GPT-5.5 is now competitive with — and in many evals ahead of — the dedicated Codex variants while staying general-purpose. OpenAI's stated direction is to let GPT-5.5 cover most agentic engineering work and reserve Codex-specific models for narrow IDE-embedded loops. - **Native computer use.** First introduced in GPT-5.4; in 5.5 it becomes the default for desktop-control agents. The model takes screenshots, emits mouse/keyboard actions, and can drive Playwright-style browser sessions inside the same call. - **Built-in web search.** Configure with `{"type": "web_search"}` in the Responses API `tools` array. The legacy `web_search_preview` still works for older integrations but doesn't expose newer filters or the `external_web_access` control. - **Lower output verbosity per task at the same accuracy.** OpenAI claims, and third-party measurements support, that GPT-5.5 finishes Codex-style tasks with fewer tokens than GPT-5.4 — a real cost reduction on agentic pipelines. - **Long cache TTL.** 24 hours by default for cached prompt prefixes, vs. minutes on earlier GPT-5 models. For RAG and agent loops with stable scaffolding this is a meaningful win. For a wider lens on how this generation lines up with peers, see our [Muse Spark vs. ChatGPT-5.4 vs. Opus 4.6 vs. Gemini 3.1 Pro analysis](https://codersera.com/blog/muse-spark-vs-chatgpt-5-4-vs-claude-opus-4-6-vs-gemini-3-1-pro/) and the [Llama 4 vs. GPT-4.5 reference](https://codersera.com/blog/llama-4-vs-gpt-45-a-comprehensive-comparison-of-the-latest-ai-models/). ## Responses API vs. Chat Completions OpenAI's official position since GPT-5.4 is that the Responses API is the default for new projects. Chat Completions is still supported, but the gap is widening with each release. Concrete differences that matter: - **Built-in tools.** Web search, file search, computer use, code interpreter, and remote MCP servers are all first-class in Responses. In Chat Completions you wire them up yourself. - **State.** Pass `previous_response_id` or use the Conversations API and the platform handles state. With Chat Completions you re-send the whole transcript, paying for it on every turn. - **Cache hit rate.** OpenAI's internal numbers show 40-80% better cache utilization on Responses vs. Chat Completions — a direct cost cut. - **SWE-bench delta.** Same prompt, same model, but Responses scores about 3 points higher on SWE-bench Verified than Chat Completions, because reasoning state is preserved between tool calls instead of being thrown away each turn. - **Tool calling at `reasoning.effort: none`.** Not supported in Chat Completions starting with GPT-5.4; works fine in Responses. - **Structured outputs.** Use `text.format` in Responses, not `response_format`. Strict mode is recommended as the default — if you've hand-rolled JSON-schema retries, delete that code. If you're maintaining a Chat Completions integration, the migration is mostly mechanical, and the cache-hit savings tend to pay for the engineering time within a few weeks of traffic. ## Tool calling, structured outputs, and the agent surface Tool calling is unchanged in shape from GPT-5.4 — JSON-schema function tools and free-form "custom tools" (for SQL, shell, config payloads) are both supported. The two practical pieces of advice: 1. **Always set `strict: true` on function tools.** The reliability gain is large, and the only reason not to is if you have legacy schemas with patterns the strict validator rejects (in which case, fix the schema). 2. **Stop describing your output schema in the system prompt.** Use Structured Outputs with a JSON Schema instead. The model adheres to it without you spending tokens explaining it, and you don't need a "validate, retry, fix" wrapper around the call. For full agent loops, the OpenAI Agent SDK (Python and TypeScript) and Codex CLI now both default to GPT-5.5 with `medium` reasoning. Codex CLI is open-source, written in Rust, and the easiest place to feel the difference between 5.5 and earlier models — the same task usually finishes in fewer turns and fewer tokens. ## How to choose: GPT-5.5, Claude Opus 4.7, or DeepSeek V4 Pro Most production teams running real volume should be using more than one of these. The hard part is the routing logic. A reasonable default: - **GPT-5.5 (medium):** the workhorse. Tool-heavy agent jobs, terminal-driven workflows, structured-output pipelines, anything where you want predictable behavior with the smallest possible token bill. - **GPT-5.5 Pro / xhigh:** reserve for genuine frontier work — research, hard math, multi-file refactors with subtle invariants. Don't put it on the hot path of a high-QPS product. - **Claude Opus 4.7:** the right choice when the task is "hold a 200k-line codebase in your head and reason about an architectural change." It still leads SWE-bench Pro by ~6 points and tends to write more idiomatic prose. The verbosity tax is real; price for it. - **DeepSeek V4 Pro:** the right choice when cost dominates. At $0.435 / $0.87 per 1M tokens in/out — the standing rate since 2026-05-22, superseding the retired $1.74 / $3.48 list price — it is about 1/11th the input price and 1/34th the output price of GPT-5.5 standard ($5 / $30), and it's within striking distance on Terminal-Bench, MCP Atlas, BrowseComp, and most knowledge-work evals. DeepSeek raises prices on **August 16, 2026** to $0.66 / $1.98 off-peak and $1.32 / $3.96 at peak, which narrows the output gap to roughly 8–15x — still decisive, but no longer 34x ([breakdown](https://codersera.com/blog/deepseek-v4-price-change-august-2026/)). Open weights. Slower on the frontier; faster on your bill. For a broader catalog of alternatives — including local models worth running yourself — see our [top 10 ChatGPT alternatives in 2026](https://codersera.com/blog/top-10-chatgpt-alternatives-2026/) and the [Qwen3.5 Omni-Plus vs. GPT-4o vs. Gemini 3.1 Pro comparison](https://codersera.com/blog/qwen3-5-omni-plus-vs-gpt-4o-vs-gemini-3-1-pro-benchmarks-pricing-and-use-cases/). ## Known limitations OpenAI's marketing won't tell you any of this, so we will: - **TTFT at high reasoning is brutal.** 115-second time-to-first-token at `xhigh` on the Responses API is not a misprint. If your product UX expects a streaming reply within five seconds, do not put `xhigh` on the hot path. - **Verbose at high effort.** The model "thinks out loud" with a lot of internal tokens. Those count as billable output. Cap with `max_output_tokens` for any user-facing flow. - **Pro tier has no cached-input discount.** If your workload has a stable preamble, this nukes one of the main reasons to keep prefixes long. Either drop to `gpt-5.5` for the prefix-heavy calls or shorten your context. - **Still loses to Claude Opus 4.7 on SWE-bench Pro.** 58.6% vs. 64.3%. If your evaluation harness is closest to "fix this real GitHub bug across 40 files," Opus is the better default. - **Web-search tool requires a reasoning model in the API.** Non-reasoning GPT-5 surfaces don't expose it the same way through the API. - **API-key auth in Codex CLI lagged the launch.** At rollout, GPT-5.5 in Codex required ChatGPT-account sign-in; API-key auth caught up shortly after but check the changelog before you wire automation. - **Computer-use is still narrow.** Best on browser-shaped UIs and well-instrumented desktop apps. Native mobile, custom-rendered canvases, and protected enterprise apps are still flaky. - **Costs explode quietly.** Reasoning tokens are billed as output, web-search tool calls are billed per call, and computer-use sessions accumulate per-screenshot costs. Instrument before you scale. ## What this means if you're hiring The GPT-5.5 generation moves AI tooling from "smart autocomplete" to "an agent that can drive your CLI, your browser, and your IDE." That changes what a senior engineer is worth — not less, but differently. You still need humans who can architect systems, write the evals, design the prompts, decide which model handles which call, and keep the agent loop from doing something stupid in production. What changes is that one such engineer, properly equipped, replaces a much larger headcount on the routine work. That's the kind of remote-ready developer Codersera vets for. If you're building agentic tooling, AI-driven backends, or just want a Python or AI engineer who already knows the difference between Responses and Chat Completions, talk to us. ## FAQ ### When was GPT-5.5 released? April 23, 2026 in ChatGPT and Codex; April 24, 2026 in the public API. ### Is there a GPT-5.5-mini? Not at launch. The mini tier is still served by `gpt-5-mini` from the GPT-5 family. OpenAI has signaled a 5.5-mini is on the roadmap but has not committed to a date. ### What is GPT-5.5 Pro? A higher-capability variant available in the API as `gpt-5.5-pro` and in ChatGPT for Pro, Business, Enterprise, and Edu plans. It's priced at $30 / $180 per 1M tokens (input/output) and does not offer a cached-input discount. ### What are the reasoning levels? `none`, `low`, `medium` (default), `high`, and `xhigh`. Higher levels burn more reasoning tokens (billed as output) and have higher TTFT but score better on hard evals. ### How big is the context window? 1,000,000 tokens for both `gpt-5.5` and `gpt-5.5-pro`. ### How does prompt caching work for GPT-5.5? Automatic on prompts of 1,024+ tokens. Cache TTL is 24 hours by default for the 5.5+ family. Cached input is billed at $0.50 per 1M (90% off) on `gpt-5.5`; the discount is not offered on `gpt-5.5-pro`. Place stable content (instructions, examples, retrieved docs) at the front of your prompt to get the most cache hits. ### Is the Batch API supported? Yes, with the standard 50% discount on most models. `service_tier: "flex"` on the Responses API gives the same 50% discount with looser SLAs. ### Should I use the Responses API or Chat Completions? Responses, for any new project. Lower cost from better caching, higher quality on tool-heavy tasks, and built-in web search, computer use, file search, and MCP. Chat Completions is in maintenance mode for new feature surface. ### Does GPT-5.5 support structured outputs? Yes — Structured Outputs with strict JSON Schema validation. Use `text.format` in Responses (not `response_format`). Set `strict: true` on function tools. ### How does GPT-5.5 compare to GPT-5 Codex? On general agentic coding (Terminal-Bench, multi-step CLI tasks) GPT-5.5 is ahead. On narrow IDE-embedded refactoring loops, GPT-5.3-Codex still has a slight edge on some coding-only averages (~63 vs. ~59 on certain coding panels). For most teams the right answer is GPT-5.5 by default and Codex models only inside the Codex IDE/CLI. ### How does GPT-5.5 compare to Claude Opus 4.7? Roughly even at the headline level: Opus leads SWE-bench Pro and architectural reasoning across large codebases; GPT-5.5 leads Terminal-Bench 2.0, agentic tool-use, and token efficiency (~72% fewer output tokens on identical coding tasks). Best production setups route between them. ### How does GPT-5.5 compare to DeepSeek V4 Pro? GPT-5.5 leads the Artificial Analysis Intelligence Index 60 to 52 and is ahead on Terminal-Bench (82.7% vs. 67.9%) and SWE-bench Pro (58.6% vs. 55.4%). DeepSeek V4 Pro is roughly 1/11th the input price and 1/34th the output price ($0.435 / $0.87 vs $5 / $30, narrowing to about 8–15x on output after DeepSeek's August 16, 2026 price rise), has open weights, and gets within a couple of points on BrowseComp and MCP Atlas. For cost-sensitive workloads or self-hosted deployments, V4 Pro is the right choice. See [our head-to-head](https://codersera.com/blog/deepseek-v4-vs-gpt-5-5-pro/) for the deep dive. ### Is GPT-5.5 good at long-context tasks? The 1M-token window is real but not free. Quality degrades on very long contexts as it does for every frontier model. For RAG, smaller, well-curated context still beats stuffing the window. Pair the long window with prompt caching and put your stable scaffolding at the front. ### Can GPT-5.5 use a computer? Yes, via the `computer_use` tool in the Responses API. It can take screenshots, control mouse and keyboard, and drive browser automation. Best on browser-shaped UIs; still flaky on protected enterprise apps and custom-rendered canvases. ### Where can I run GPT-5.5 locally? You can't — it's API-only and OpenAI does not publish weights. If you need on-prem or air-gapped, the closest open-weight options are DeepSeek V4 and the Qwen3.5 family. ## Next steps If you're shipping with GPT-5.5, the bottleneck is rarely the model — it's the engineer who knows how to wire Responses, structured outputs, prompt caching, tool calling, and a sane fallback policy into a system that doesn't fall over at 3am. [Hire a Codersera-vetted Python or AI engineer](https://codersera.com/blog/hire/hire-python-developer) who has already built one. =============================================================== [4/20] Llama 4: The Complete Developer Guide (2026) URL: https://codersera.com/blog/llama-4-complete-guide-2026/ Last updated: 2026-08-13 =============================================================== **Quick answer (refreshed May 27, 2026).** Three big things have changed in the Llama story since this guide shipped. (1) **Behemoth is effectively shelved** — never publicly released, never formally cancelled; mid-training MoE-routing and chunked-attention issues at 2T-scale meant Meta lost confidence the gains justified shipping. (2) On April 8, 2026, **Meta Superintelligence Labs (MSL) shipped Muse Spark**, a *closed-weight, API-only* reasoning model — Meta's first proprietary frontier release. Index 52 on Artificial Analysis (4th, behind GPT-5.4, Gemini 3.1 Pro, Claude Opus 4.6), and leads HealthBench Hard at 42.8 vs Gemini 3.1 Pro's 20.6. (3) **Llama 5 ("Avocado")** — built for world-model reasoning targeting 3rd-gen Ray-Ban Meta glasses — has slipped from a Q1 2026 leak to a Goldman/Finterra 2027 forecast; Polymarket consensus pricing puts a 2026 ship below 20%. **For anyone choosing an open-weight stack, Llama 4 Scout and Maverick remain Meta’s only large open-weight models** — but they are no longer Meta’s last word on open weights: on August 10, 2026 Meta shipped **Muse Glimmer**, a 30B Apache-2.0 open-weights model, and announced that Muse Spark 1.2’s weights will follow. Scout for long-context retrieval; Maverick for long-context generation. Coding-first workloads should look outside Llama — DeepSeek V4 Pro and Qwen 3.6-27B both materially beat Maverick. 🆕 **Update (August 13, 2026):** The Llama-to-Muse arc now has a third act. Meta first pivoted away from open-weight Llama to its closed, proprietary [Muse Spark](https://codersera.com/blog/muse-spark-complete-guide-2026/) line (Meta Superintelligence Labs, April 2026), then shipped the [Muse Code](https://codersera.com/blog/muse-code-complete-guide-2026/) terminal agent on Muse Spark 1.2 (August 5, 2026) — and then **went back to open weights** on August 10, 2026 with **Muse Glimmer**, a 30B Apache-2.0 model, plus an announcement that Muse Spark 1.2’s own weights will be open-sourced. **Llama 4 is no longer Meta’s last open release.** ## What's new in the Llama story since April 2026? 1. **Behemoth status: effectively shelved.** The ~2T-total / 288B-active / 16-expert "teacher" model previewed in April 2025 has still not shipped publicly, with no formal cancellation. Root causes reported in 2025: a mid-training MoE-routing switch disrupted expert specialization, and the chunked-attention scheme introduced blind spots at chunk boundaries that hurt long-form reasoning at 2T scale. The window to ship Behemoth as Meta's flagship effectively closed when Muse Spark launched. 2. **Meta Muse Spark (April 8, 2026) — Meta's first closed-weight frontier model.** Native multimodal reasoning with tool-use, visual chain-of-thought, multi-agent orchestration. Built by Meta Superintelligence Labs under **Alexandr Wang** (ex-Scale AI; hired as Meta's first Chief AI Officer in a reported $14B+ talent-and-equity deal). Available in `meta.ai`, the Meta AI app, WhatsApp / Instagram / Facebook / Messenger / Ray-Ban Meta glasses, and a private-preview API. *No weights, no architecture paper.* 3. **Meta went open again — August 10, 2026.** Meta Superintelligence Labs released **Muse Glimmer**: a 30B dense decoder-only multimodal model (built-in vision encoder) under **Apache 2.0**, 128K context, text + image in, 100+ languages, distilled from Muse Spark. Weights are ungated on Hugging Face (~121k downloads on the base repo, ~352k on Unsloth’s GGUF within days); at ~4-bit the language model fits under 20 GB, so it runs on one 24–32 GB consumer GPU or a Mac. Zuckerberg separately announced that **Muse Spark 1.2’s weights will be open-sourced** — *announced, not shipped*; nothing had been published as of August 13, 2026. Honest read on quality: on Meta’s own size-class chart, Qwen3.6-27B beats Glimmer on four of seven rows (SWE-Bench Verified 77.2 vs 76.0, TerminalBench 2.1 60.7 vs 51.7, OSWorld-Verified 75.6 vs 65.9, SkillsBench 46.6 vs 44.3), while Glimmer takes the tool-calling rows (MCP Atlas 75.5 vs 62.5, SWE-Bench Pro 51.2 vs 50.2, WildClawBench 47.6 vs 43.2). It is a tool-use specialist in the 30B class, not a general coding leader. 4. **Llama 5 codename "Avocado" — 2027 most likely.** Earlier 2025 leaks pointed at Q1 2026; that obviously slipped. Goldman Sachs and Finterra now forecast 2027, citing stability priorities, EU AI Act compliance overhead, and data-availability concerns. Polymarket has 102 live Llama-5 markets as of May 19, 2026 with consensus below 20% odds on a 2026 ship. 5. **Scout's 10M context — real for retrieval, weak for synthesis.** Independent testing has caught up with Meta's claim. Scout scores **15.6% on Fiction.LiveBench at 128K tokens vs Gemini 2.5 Pro's 90.6%** — recall works for needle-in-haystack but collapses on tasks that need to synthesise across buried tokens. At quantization levels needed for consumer GPUs, 4-bit perplexity rises 15-20% past 5M tokens — effective window is closer to 5M than 10M. Unquantized Scout past 1.4M tokens needs 8× H100s. 6. **Maverick is no longer competitive at the open coding frontier.** SWE-bench Verified: Maverick ~24%, Llama 4 Scout slightly better, but DeepSeek V4 Pro hits 80.6% and Qwen 3.6-27B 77.2%. For code generation, agentic refactoring, or repo-level work, reach for DeepSeek V4 Pro or Qwen 3.6 instead. 7. **Licensing reminder that bites harder now.** Llama 4 Community License: commercial OK below 700M MAU, special-license required above. The **EU multimodal carve-out still applies** — vision capability off-limits to EU-domiciled entities pending AI Act clarity; text-only paths fine. Muse Spark closes neither gap (API-only, outside the open-weight ecosystem entirely). ## Where does each Llama 4 variant fit now? | Use case | Best Llama option | Or look outside Llama for | | --- | --- | --- | | Long-context retrieval (RAG, code repo scan) | **Llama 4 Scout** (10M nominal / 5M effective) | Gemini 3.5 Pro (2M with stronger recall) | | Long-context generation (whole-doc rewrites) | **Llama 4 Maverick** (1M context) | Claude Opus 4.7 (1M with extended thinking) | | Coding / agentic refactoring | — | **DeepSeek V4 Pro** or **Qwen 3.6-27B** | | EU-deployed text-only chat | Llama 4 Maverick (text-only path) | Mistral Medium 3.5 (EU-friendly, fully open) | | Closed-weight Meta API surface | — | **Muse Spark** (closed; private preview) | | Small open-weight Meta model on one GPU | — | **Muse Glimmer 30B** (Apache 2.0, 128K context, under 20 GB at 4-bit) | Cross-references for the broader picture: our [open-source LLMs landscape pillar](https://codersera.com/blog/open-source-llms-landscape-2026/) for the May 2026 open-weights leaderboard, [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) for the current open coding leader, [Qwen guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) for the 3.6 / 3.7 Max story, and [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) for running Llama 4 locally on Mac. *Last updated: May 1, 2026* Llama 4 is the open-weight model family that forced every other lab to publish a serious mixture-of-experts checkpoint. It is also the model family that came with the loudest licensing footnote, the most public benchmark controversy of the cycle, and a 10-million-token context window that nobody else has matched. If you are planning to ship a product on top of Llama 4 in 2026, the practical questions are not "is it good?" but "which variant, which provider, under which license, against which alternative." This guide answers those questions for engineering leaders, ML platform teams, and developers who have to deploy, fine-tune, or pay for Llama 4 in production. It pulls together Meta's own model cards, the Llama 4 Community License, current hosted-provider pricing, public benchmarks (including the ones that look bad for Llama 4), self-hosting hardware tiers, and a frank read on when Llama 4 is the right choice and when it is not. ## TL;DR - **The herd:** Three variants, all MoE. *Scout* (109B total / 17B active / 16 experts / 10M context). *Maverick* (≈400B total / 17B active / 128 experts / 1M context). *Behemoth* (≈2T total / 288B active / 16 experts) was previewed but never publicly released. - **Architecture:** Native multimodal (text + image, early fusion), interleaved attention with NoPE layers ("iRoPE") to push context length, MoE routing for efficient inference at frontier scale. - **License:** Llama 4 Community License — commercial use allowed below 700M MAU, with EU multimodal restrictions and an Acceptable Use Policy. Not OSI-open, not Apache, not MIT. - **Reality check:** Public benchmarks and community testing put Llama 4 Maverick behind DeepSeek V4 and Qwen 3.5/3.6 on coding and hard reasoning. The LMArena "experimental" submission controversy (April 2025) was real and changed how the model is perceived. - **Where it wins:** Long-context retrieval (Scout), multimodal vision in an open-weight model, hosted price-performance on Groq, and a proven self-hosting story with vLLM and SGLang. - **Where it loses:** SWE-bench-style end-to-end coding agents, regulated EU deployments needing the vision modality, and any project that wants a clean MIT/Apache license. ## 1. The Llama 4 family at a glance Meta released Llama 4 on April 5, 2025. Unlike Llama 2 and Llama 3, the entire Llama 4 family is mixture-of-experts. That is the single biggest architectural change. Instead of running every parameter on every token, an MoE model routes each token to a small subset of "expert" sub-networks. Total parameter count goes up; active parameter count per token stays small. The practical effect: Llama 4 Maverick has roughly 400B parameters on disk but only ~17B active per forward pass, so its inference cost behaves like a 17B model while its capacity behaves like a much larger one. That is the trade Meta made. | Variant | Total params | Active params | Experts | Context | Multimodal | Status | | --- | --- | --- | --- | --- | --- | --- | | Llama 4 Scout | 109B | 17B | 16 | 10M tokens | Text + image | Released | | Llama 4 Maverick | ≈400B | 17B | 128 | 1M tokens | Text + image | Released | | Llama 4 Behemoth | ≈2T | 288B | 16 | — | Text + image | Preview only, not released | Both released variants were pretrained on roughly 22 trillion tokens of mixed text, image, and video data, with native multimodality from day one (early fusion of text and vision tokens into a unified backbone, rather than a vision encoder bolted onto a frozen LLM). ## 2. Architecture: MoE, iRoPE, and the 10M context claim Three architectural choices matter for engineers deciding whether Llama 4 fits: ### Mixture-of-Experts routing Scout uses a "full" MoE pattern across its layers. Maverick uses an alternating dense/MoE layout — experts are applied in roughly half the layers, with dense layers in between. This matters for inference frameworks: vLLM and SGLang support both layouts, but the active-parameter advertising is misleading if you are sizing GPU memory. You still need to load all expert weights into VRAM (or stream them, which is slow). ### iRoPE for long context Llama 4 interleaves *NoPE* layers (no positional encoding, full causal attention over the entire context) every fourth layer with three *RoPE* layers using chunked attention, plus inference-time temperature scaling on attention. Meta calls this iRoPE. The result is the 10M-token context on Scout, with what Meta reports as perfect needle-in-the-haystack retrieval across that range. Independent testing (Andri.ai, dev.to community runs) confirms strong retrieval but flags a separate issue: precision degrades for tasks that require *reasoning* over the long context, not just retrieval. A 10M-token retrieval window is not the same as a 10M-token reasoning window. If you are pushing a full monorepo into the prompt to ask a refactoring question, expect uneven results. ### Native multimodality The vision encoder is a Meta-trained variant of MetaCLIP, jointly trained with a frozen Llama backbone so the encoder produces tokens the LLM can natively consume. Llama 4 has been validated for up to five input images per prompt. This is competitive with GPT-4o and Gemini 2.0 Flash on standard vision benchmarks and is the main reason teams keep Llama 4 in the running for multimodal use cases despite the licensing friction. ## 3. License: read this before you ship Llama 4 ships under the **Llama 4 Community License Agreement**. Commercial use is permitted, but it is not OSI-approved open source. The clauses that actually matter: - **700M MAU threshold.** If your product or service had more than 700 million monthly active users in the calendar month before Llama 4's release (April 2025), you must request a separate license from Meta, granted at Meta's "sole discretion." This is the same clause that has been in every Llama license since Llama 2 and is aimed squarely at hyperscaler competitors. - **EU restriction on multimodal.** The Llama 4 multimodal models cannot be used by, or distributed to, individuals or companies "domiciled in" the EU. This is Meta's response to AI Act ambiguity. The text-only paths are not blocked, but the vision capability — the headline feature — is off-limits for EU-based deployments without bespoke arrangements. - **Acceptable Use Policy.** Standard restrictions on illegal use, weapons development, CSAM, election interference, and so on. Read it; it is short. - **Attribution.** Distributions of Llama 4 or fine-tunes must include "Built with Llama" attribution and a copy of the license. If your legal team requires an Apache 2.0 / MIT model, Llama 4 is out. [DeepSeek V4 is MIT-licensed](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) and is the most direct frontier-class alternative in that case. ## 4. Benchmarks: what is real, what is marketing Meta's launch deck claimed Maverick beats GPT-4o and Gemini 2.0 Flash on a broad range of benchmarks. The community spent April 2025 stress-testing those claims. The summary: Llama 4 is a competent frontier model, but on the benchmarks engineers actually care about — coding, hard reasoning, agentic tool use — it is not the leader. | Benchmark | Llama 4 Scout | Llama 4 Maverick | DeepSeek V4 | Qwen 3.6 | Notes | | --- | --- | --- | --- | --- | --- | | MMLU-Pro | ~74 | ~80 | 92.8 | ~88 | Knowledge + reasoning | | GPQA Diamond | ~57 | ~70 | ~82 | 86.0 | PhD-level science | | LiveCodeBench | ~32 | ~43 | ~62 | ~55 | Contamination-resistant coding | | SWE-Bench Verified | ~14 | ~24 | ~55 | ~49 (Pro) | End-to-end repo bug fixes | | Needle-in-Haystack (10M) | ≈100% | n/a (1M) | n/a | n/a | Long-context retrieval | Numbers are approximate and rounded from public reports (Meta model card, ArtificialAnalysis Intelligence Index, llm-stats, Composio, Spheron benchmarks). Treat them as directionally correct, not exact. Where a model has both reasoning and non-reasoning modes, scores reflect the best non-reasoning single-pass mode for fairness. The pattern is consistent: Llama 4 holds its own on knowledge and long-context retrieval, but trails DeepSeek V4 and Qwen 3.5/3.6 on coding and hard reasoning. The [comparison with Claude 3.7 Sonnet](https://codersera.com/blog/llama-4-vs-anthropic-claude-3-7-sonnet/) and the [comparison with GPT-4.5](https://codersera.com/blog/llama-4-vs-gpt-45-a-comprehensive-comparison-of-the-latest-ai-models/) tell the same story for closed-source frontier models. ### The LMArena episode Worth noting because it shapes how the market reads any Llama 4 benchmark today. In April 2025, Meta submitted "Llama-4-Maverick-03-26-Experimental" to LMArena — a variant tuned for human-preference voting, distinct from the public release weights. It topped the leaderboard. LMSYS later acknowledged the variant was not labeled clearly enough, and the public release of Maverick performs noticeably worse on the same arena. Meta's VP of GenAI denied training on test sets. The community read it as benchmark gaming regardless. Practical impact: discount any Llama 4 chart that cites a single LMArena number, and look at code- and reasoning-specific benchmarks instead. ## 5. Hosted pricing: where to actually run it Llama 4 is broadly available across the hosted-inference ecosystem. The price spread is significant — Bedrock and Azure cost roughly 3-5x what specialty inference shops charge. Numbers below are public list prices as of late April 2026, per million tokens. | Provider | Scout (in / out) | Maverick (in / out) | Notes | | --- | --- | --- | --- | | Groq | $0.11 / $0.34 | $0.50 / $0.77 | Fastest tokens/sec; LPU hardware | | Together AI | $0.18 / $0.59 | $0.27 / $0.85 | Mature, broad model catalog | | DeepInfra | ~$0.08 / $0.30 | ~$0.20 / $0.60 | Cheapest blended price | | Fireworks AI | ~$0.15 / $0.60 | ~$0.22 / $0.88 | Strong fine-tune hosting | | AWS Bedrock | ~$0.17 / ~$0.66 | ~$0.35 / $0.80 | IAM-native, expensive at scale | | Vertex AI (GCP) | Listed via Model Garden | Listed via Model Garden | Pricing tracks Bedrock | | Azure AI Foundry | Listed | Listed | 15–40% over direct API pricing | If you only care about price, DeepInfra and Groq are the floor. If you care about latency on Maverick, Groq's LPU is unmatched for short prompts. If you need an enterprise audit trail, Bedrock or Azure is what your procurement team will accept. If you need fine-tune hosting, Fireworks and Together are the practical choices. ## 6. Self-hosting: hardware tiers and serving frameworks You self-host Llama 4 for one of three reasons: data residency, per-token cost at very high volume, or fine-tune deployment without sending weights to a vendor. Hardware sizing depends on quantization. ### Hardware tiers - **Scout, Q4 quantized:** Single 24-48GB GPU (RTX 4090 / 6000 Ada / A6000) for usable throughput; 8GB VRAM is theoretically possible at the most aggressive quants but not production-grade. ~16GB system RAM minimum. - **Scout, FP8/FP16:** Single H100 80GB or 2x A100 40GB. The "single H100" claim Meta makes is technically true at FP8. - **Maverick, Q4 quantized:** 2-4x H100 80GB or equivalent. The full expert weights still have to live in memory. - **Maverick, FP16:** 8x H100 node (640GB HBM total). This is the production target. Detailed install walkthroughs by OS: [Ubuntu](https://codersera.com/blog/how-to-run-llama-4-on-ubuntu-a-comprehensive-guide/), [macOS](https://codersera.com/blog/running-llama-4-on-mac-an-installation-guide/), [Windows](https://codersera.com/blog/running-llama-4-on-windows-an-installation-guide/). For comparison with the smaller-footprint open model, see [Gemma 4 vs Llama 4 local deployment](https://codersera.com/blog/gemma-4-vs-llama-4-local-deployment-2026/). ### Serving framework choice - **vLLM** — the default. Mature MoE support, broad community, easy OpenAI-compatible server. Use this unless you have a reason not to. - **SGLang** — better for shared-context workloads (chat, RAG, agents) thanks to RadixAttention; community reports up to ~29% throughput gains over vLLM in shared-prefix scenarios. - **TGI (Hugging Face)** — now in maintenance mode; HF themselves point new users at vLLM or SGLang. Avoid for new deployments. - **Ollama** — for laptops, dev machines, and small-team prototypes. Not a production serving stack. ## 7. Fine-tuning Llama 4 Three viable paths: - **Unsloth** — currently the only stack with working 4-bit QLoRA for Llama 4 Scout, with claimed ~1.5x speedup and ~50% VRAM savings versus Flash Attention 2 baselines. The right choice if you are tuning on a single 80GB card. - **torchtune** — Meta's first-party PyTorch library. Supports full fine-tunes, LoRA, QLoRA, and RLHF/RLVR. Best when you want minimal abstraction and you trust your own infra. - **Axolotl / LlamaFactory** — config-driven, multi-GPU friendly, broad model coverage. Good for teams running tunes across many model families. For Maverick, expect to need at least an 8x H100 node for any meaningful tune — even with QLoRA, the expert tensors are large. Most teams who fine-tune Llama 4 are tuning Scout and serving Maverick stock. ## 8. How Llama 4 stacks up against the rest of the open-weight field | Model | License | Coding | Reasoning | Long context | Multimodal | Best for | | --- | --- | --- | --- | --- | --- | --- | | Llama 4 Maverick | Llama 4 Community | Mid | Mid | 1M | Yes | Multimodal apps, broad ecosystem support | | Llama 4 Scout | Llama 4 Community | Mid | Mid | 10M | Yes | Long-document retrieval, single-GPU serving | | Muse Glimmer 30B | Apache 2.0 | Mid | Mid | 128K | Yes | Tool-calling / MCP agents on a single GPU | | DeepSeek V4 | MIT | Top tier | Top tier | 128K | Limited | Coding, math, frontier reasoning | | Qwen 3.6 (35B-A3B) | Apache 2.0 | Top tier (sub-40B) | Top tier (sub-40B) | 1M | Yes | Best price/perf in the small-MoE class | | Gemma 4 31B | Gemma terms | Mid | Above Scout on GPQA | 128K | Yes | On-device, edge, single-GPU inference | | Mistral Medium 3.5 | Mistral commercial | Mid | Mid | 128K | Limited | EU-friendly hosted serving | Detailed pairwise reads: [Llama 4 vs Mistral 7B](https://codersera.com/blog/llama-4-vs-mistral-7b-a-comprehensive-comparison-of-ai-models/), [Gemma 4 vs Llama 4](https://codersera.com/blog/gemma-4-vs-llama-4-local-deployment-2026/), [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). ## 9. Known limitations - **Coding gap.** Maverick trails DeepSeek V4 and Qwen 3.6 on SWE-bench Verified and LiveCodeBench by a meaningful margin. If your primary workload is code generation, agentic refactoring, or repo-level bug fixing, Llama 4 is not the strongest open-weight choice in 2026. - **Long-context reasoning, not just retrieval.** Scout's 10M context is real for needle-in-haystack tasks but degrades on tasks that require chained reasoning over the full window. Test on your actual workload before committing. - **EU multimodal restriction.** Vision is unavailable for EU-domiciled licensees. This is a hard block for many EU-based products. - **700M MAU clause.** Not a problem for most companies. A serious problem if you are a hyperscaler or social platform. - **License is not OSI-open.** No "Open Source" claim, no Apache/MIT permissiveness, attribution requirements on derivatives. - **LMArena trust deficit.** The April 2025 "experimental" submission episode means leaderboard scores for Llama 4 are read with extra skepticism. Use task-specific benchmarks instead. - **MoE memory tax.** "17B active" is not the same as "17B model." You still need to load all experts. Plan VRAM for total parameters, not active. - **Tool-use and function-calling reliability.** Community testing reports inconsistent JSON adherence and tool-call formatting compared with Claude 3.7 and DeepSeek V4. Heavy agent stacks may need extra guardrails. ## 10. When to choose Llama 4 Choose Llama 4 if at least two of the following are true: - You need **open-weight multimodal** with text + image in the same model and you are not EU-domiciled. - You need a context window beyond 1M tokens, and your task is retrieval-shaped (find this clause in this contract) rather than reasoning-shaped (synthesize an argument across the entire contract). - Your inference budget is tight and you want Groq-class latency or DeepInfra-class price-per-token without operating your own GPUs. - Your stack is already on Meta's ecosystem (Llama Stack, torchtune, Meta-trained MetaCLIP) and you want continuity. - You want a battle-tested open-weight base for fine-tuning, with strong tooling (Unsloth, torchtune, Axolotl, LlamaFactory) and broad hosted-fine-tune support. Skip Llama 4 if your top priority is frontier coding (DeepSeek V4 or Qwen 3.6 win), if you require Apache/MIT licensing (DeepSeek V4, Qwen), or if you are an EU-domiciled team that needs the vision modality. ## FAQ ### Is Llama 4 free to use commercially? Yes, with conditions. The Llama 4 Community License permits commercial use for licensees with fewer than 700M MAU as of April 2025. Above that threshold you must request a separate license from Meta. Attribution ("Built with Llama") is required on derivatives. ### Can I use Llama 4 in the European Union? The text-only paths are usable, but the multimodal (vision) capabilities are excluded for EU-domiciled licensees under the current license terms. If you need image input in the EU, look at Qwen 3.5/3.6 VL, Mistral, or hosted closed-source models instead. ### What is the difference between Llama 4 Scout and Maverick? Both have 17B active parameters per token. Scout is 109B total with 16 experts and a 10M-token context. Maverick is ≈400B total with 128 experts and a 1M-token context. Maverick is the higher-capacity model; Scout is the long-context specialist. ### Was Llama 4 Behemoth ever released? No. Behemoth (≈2T parameters, 288B active, 16 experts) was previewed at the April 2025 launch as still in training. As of May 2026, public weights have not shipped. ### Is Llama 4 still Meta’s only open-weight model? No. On August 10, 2026 Meta released **Muse Glimmer** — a 30B dense multimodal model under Apache 2.0, with a 128K context and ungated weights on Hugging Face. It is far smaller than Llama 4 Maverick and is built to run locally (under 20 GB at ~4-bit on one consumer GPU or a Mac), so it does not replace Scout’s 10M context or Maverick’s capacity — but it does end the reading that Llama 4 was Meta’s last open release. Meta has also announced that the closed Muse Spark 1.2’s weights will be open-sourced; as of August 13, 2026 those weights had not shipped. ### How does Llama 4 Maverick compare to GPT-4o? On Meta's launch benchmarks, Maverick edged GPT-4o on several multimodal and long-context tasks. On independent code and reasoning benchmarks (LiveCodeBench, SWE-Bench Verified), GPT-4o and successor closed models remain ahead. See the [Llama 4 vs GPT-4.5 comparison](https://codersera.com/blog/llama-4-vs-gpt-45-a-comprehensive-comparison-of-the-latest-ai-models/) for the head-to-head. ### How does Llama 4 compare to DeepSeek V4? DeepSeek V4 wins on coding (LiveCodeBench, SWE-Bench Verified), hard reasoning (GPQA, MMLU-Pro), and licensing (MIT vs Llama Community). Llama 4 wins on multimodality, ecosystem maturity, and Scout's 10M context. Pick DeepSeek for code agents; pick Llama for multimodal apps. ### What hardware do I need to run Llama 4 Scout locally? A single 24GB consumer GPU runs Scout at Q4 quantization. A single H100 80GB runs it at FP8. For full FP16 you want 2x A100 80GB or equivalent. See the OS-specific guides for [Ubuntu](https://codersera.com/blog/how-to-run-llama-4-on-ubuntu-a-comprehensive-guide/), [macOS](https://codersera.com/blog/running-llama-4-on-mac-an-installation-guide/), and [Windows](https://codersera.com/blog/running-llama-4-on-windows-an-installation-guide/). ### Which hosted provider has the cheapest Llama 4? DeepInfra typically has the lowest blended per-token price. Groq has the lowest latency. Together AI is the best balance of price, latency, and feature coverage. Bedrock and Azure are 3-5x the dedicated-inference shops but are the realistic choices when you need IAM, VPC, or enterprise audit. ### Is the 10M context window real? Real for retrieval tasks (needle-in-haystack) — Scout achieves near-perfect retrieval across 10M tokens. Less reliable for tasks that require chained reasoning over that full window. Validate on your workload before committing to the 10M context as a product feature. ### Should I fine-tune Scout or Maverick? Fine-tune Scout for domain adaptation, instruction tuning, and downstream tasks where 17B active and 109B total is sufficient. Use Maverick stock — fine-tuning it requires multi-node H100 infrastructure that most teams will not invest in. Unsloth is the most accessible Scout fine-tune path; torchtune is the best path if you want first-party tooling. ### What was the LMArena controversy? Meta submitted a variant called "Llama-4-Maverick-03-26-Experimental" — tuned for human-preference voting and distinct from the public release weights — to LMArena. It topped the leaderboard. LMSYS later acknowledged the labeling was not sufficiently clear, and the released Maverick performs noticeably worse on the same arena. Treat single-number leaderboard claims for Llama 4 with skepticism. ### Does Llama 4 support function calling and tool use? Yes, both Scout and Maverick support tool calling, and the model card documents the prompt format. Community reports note inconsistent JSON adherence compared with Claude 3.7 or DeepSeek V4, so production agent stacks should add validation and retry logic. ### Can I run Llama 4 on a Mac? Scout runs on Apple Silicon Macs with 32GB+ unified memory at Q4 quantization, via Ollama or llama.cpp. Maverick is impractical on consumer Apple hardware due to total parameter count. See [Running Llama 4 on Mac](https://codersera.com/blog/running-llama-4-on-mac-an-installation-guide/). ### Is Llama 4 a good choice for a coding agent? Not the best choice in 2026. DeepSeek V4 and Qwen 3.6 lead on SWE-bench Verified and LiveCodeBench. Llama 4 is competent for general code completion and explanation but trails on agentic, repo-level tasks. If coding is the primary workload, choose accordingly. ## Next steps Llama 4 in production is a real engineering project: license review, provider selection, hardware sizing if you self-host, fine-tuning if you have proprietary data, and evaluation on your actual workload rather than launch-deck benchmarks. Most teams underestimate the evaluation work and overestimate how much the headline numbers transfer to their domain. If you need senior Python and ML engineers who have shipped Llama, DeepSeek, or Qwen workloads in production — including vLLM/SGLang serving, QLoRA fine-tunes, and multimodal pipelines — Codersera matches you with vetted, remote-ready developers in days, not months. [Hire a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer) and extend your team with someone who already knows the trade-offs in this guide. =============================================================== [5/20] Qwen 3.5: The Complete Developer Guide (2026) URL: https://codersera.com/blog/qwen-3-5-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer (refreshed May 27, 2026).** Qwen 3.5 is no longer the current generation. Alibaba shipped **Qwen 3.6** in two waves in April (the 35B-A3B MoE on April 16, then a fully-dense **Qwen 3.6-27B** on April 22 that beats the 397B-A17B 3.5 flagship on agentic coding while running on a single consumer GPU), then announced **Qwen 3.7 Max** on May 20 at the Alibaba Cloud Summit — a 1M-context, reasoning-native, **closed-weight** DashScope-API-only flagship priced at $2.50 in / $7.50 out per 1M tokens (90% cached discount). The family now cleanly splits into open-weight workhorses (3.5 + 3.6 under Apache 2.0) and a closed-weight frontier model (3.7 Max). For local self-hosting, Qwen 3.6-27B is the new default best local coding model; for big agentic-coding workloads at API price, 3.7 Max beats GPT-5.5 on SWE-Pro / Terminal-Bench at ~1/4 the cost. **Qwen3-Coder-30B-A3B** remains the MLX-on-Mac favourite; no 3.6 or 3.7 Coder variant has shipped yet. ## What's new in the Qwen lineup since February 2026? 1. **Qwen 3.6-35B-A3B (April 16, 2026)** — refreshed MoE, 35B total / ~3B active, 262K native context extendable to ~1M via YaRN, Apache 2.0, natively multimodal. 2. **Qwen 3.6-27B (April 22, 2026)** — the more interesting release. Fully-dense 27B (every parameter fires on every token), **beats Qwen 3.5's 397B-A17B flagship** on coding suites: SWE-bench Verified 77.2 vs 76.2, Terminal-Bench 2.0 59.3 vs 52.5, SWE-bench Pro 53.5 vs 50.9. Fits in 16.8 GB at Q4_K_M; single consumer GPU. The new **"best local model for agentic coding"** recommendation. Apache 2.0. 3. **Qwen 3.7 Max (May 20, 2026)** — the frontier flagship and the cleanest break in the lineup. 1M-token context (third-party recall tests hold up well past 800K), native extended-thinking mode, Anthropic Messages protocol compatibility as a drop-in. **Closed-weight, DashScope API-only** — no GGUF, no Hugging Face checkpoint. $2.50 in / $7.50 out per 1M tokens with 90% cached-input discount. No timeline announced for an open-weight 3.7 variant. 4. **Qwen3-Coder-30B-A3B** is *still* the MLX-on-Mac coding default in May 2026 — no Qwen3.6-Coder or Qwen3.7-Coder has shipped. 4-bit / 5-bit / 6-bit / 8-bit MLX quants on Hugging Face under `mlx-community` and `lmstudio-community`; wired into LM Studio, Ollama, llama.cpp, Unsloth. Don't downgrade to Qwen2.5-Coder. 5. **Multimodal:** 3.6-27B and 3.6-35B-A3B are both natively multimodal (image + text), covering most VLM needs without dropping to a separate Omni checkpoint. **Qwen3-Omni-30B-A3B** remains the current end-to-end multimodal flagship (text + image + audio + video in, text + speech out; SOTA on 32 of 36 audio/audio-visual benchmarks). ## Where does Qwen 3.7 Max actually win? | Benchmark | Qwen 3.7 Max | Claude Opus 4.7 | GPT-5.5 | | --- | --- | --- | --- | | SWE-bench Pro | **60.6** | 58.x | 57.x | | Terminal-Bench 2.0 | **69.7** | 66.x | 71.x* | | GPQA Diamond | 92.4 | **94.x** | 93.x | | HumanEval | 94.5 | **96.x** | 95.x | | HMMT 2026 Feb (math) | **97.1** | 95.x | 96.x | | $ / 1M tok (in / out) | **$2.50 / $7.50** | $15 / $75 | $10 / $30 | Honest read: **Opus 4.7 still leads pure reasoning** (GPQA, HLE, AIME), **GPT-5.5 still leads on browser/OS agent benchmarks**, but **Qwen 3.7 Max wins agentic coding** at roughly 1/6 the cost of Opus 4.7 and 1/4 the cost of GPT-5.5. If you're running long-horizon coding agents at volume, the price-performance gap is the headline — not the raw benchmark deltas. See our [Claude Opus 4.7 guide](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/), [GPT-5.5 guide](https://codersera.com/blog/gpt-5-5-complete-guide-2026/), and [AI coding agents complete guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) for the broader comparison. For the deep-dive on the rest of the Qwen 3.5 family and how to deploy it, read on. *Last updated: May 1, 2026.* Qwen 3.5 is the most quietly important open-weight release of the year. While the discourse has fixated on whether GPT-5.2 or Claude Opus 4.6 wins this week's Arena slot, Alibaba's Qwen team has shipped a full, eight-tier model family under Apache 2.0, paired with a hybrid Gated DeltaNet architecture, and undercut almost every closed-source competitor on cost-per-token by an order of magnitude. For engineering teams that care about cost discipline, self-hosting, and avoiding lock-in, Qwen 3.5 is the model family to understand in 2026. This guide is the long version. We cover the full Qwen 3.5 family from the 0.8B edge model up to the 397B-A17B flagship and the Omni-Plus multimodal variant; the architecture and license; real benchmark scores against DeepSeek V4, Llama 4, and Gemma 4; pricing across DashScope and the major hosted providers; hardware requirements for self-hosting; how Qwen 3.5 plugs into Claude Code as a coding agent; and the limitations you need to know before staking production on it. Qwen lineup status — May 2026 | Release | Status as of May 2026 | | --- | --- | | **Qwen 3.6** | Shipped. Open weights on Hugging Face (Apache 2.0 for the open variants). Lineup includes a 27B dense model, a 35B-A3B MoE, and the hosted Qwen3.6 Plus tier. **Run this today** — see [how to run Qwen 3.6 locally](https://codersera.com/blog/how-to-run-qwen-3-6-locally-2026/). | | **Qwen 3.7-Max** | Officially announced May 20, 2026 at the Alibaba Apsara Summit; API rolling out on Alibaba Cloud Model Studio. Preview variants (Max-Preview text, Plus-Preview vision) are live on chat.qwen.ai and lmarena.ai. **No Hugging Face weights yet.** Headline claim: 35-hour autonomous run with 1000+ tool calls (Alibaba-reported). LM Arena (neutral): Max-Preview ranks #13 overall and #7 on Math; Plus-Preview puts Alibaba as the #5 lab in Vision Arena. Full breakdown in [Qwen 3.7 release date and what’s new](https://codersera.com/blog/qwen-3-7-release-date-whats-new-2026/). | | **Qwen 3.5** | Prior generation. Superseded for new work; kept here as the canonical reference for the family. | This pillar updates as the family evolves; the 3.7 page above tracks open-weight availability in place. ## TL;DR - **The family** spans Qwen3.5-0.8B, 2B, 4B, 9B, 27B (dense), 35B-A3B and 122B-A10B (MoE), and the Qwen3.5-397B-A17B flagship, plus the Qwen3.5-Omni-Plus multimodal variant. All open-weight under Apache 2.0 except the hosted-only Plus tier. - **Architecture** is a hybrid: roughly 75% Gated DeltaNet linear-attention layers and 25% full softmax attention with GQA/RoPE, plus sparse MoE on the larger tiers. This is what gets you 256K native context with sane KV cache cost. - **Benchmarks**: the 397B-A17B flagship hits 88.4 GPQA Diamond, 91.3 AIME 2026, 83.6 LiveCodeBench v6, and 86.7 Tau2-Bench (agents). The 9B model scores 81.7 on GPQA Diamond on a laptop. - **Pricing** on DeepInfra runs $0.01 / $0.05 per 1M tokens for the 0.8B, scaling to $0.54 / $3.40 per 1M for the 397B-A17B. DashScope offers an Anthropic-API-compatible endpoint that drops directly into Claude Code. - **Cost-per-quality** is the headline. Qwen has been quietly winning in production because the 35B-A3B and 9B tiers hit 80-90% of frontier-model accuracy at 1-5% of the cost, with permissive licensing and self-hosting on commodity GPUs. - **The catch**: ecosystem maturity around fine-tuning and agent tooling lags Llama, the Plus and Max tiers are not open-weight, and the Omni-Plus real-time speech features still have rough edges in non-English languages. ## The Qwen 3.5 family at a glance Qwen 3.5 was released in three waves between February 16 and March 2, 2026. Unlike Qwen 3, which split text models from VL (vision-language) models into separate trees, Qwen 3.5 unifies them: a single backbone trained with early fusion of text and multimodal tokens. In Alibaba's own evaluations, this unified approach matches or beats the separate Qwen3-VL line on visual benchmarks while keeping text performance intact. | Model | Total params | Active params | Architecture | Context | License | Best for | | --- | --- | --- | --- | --- | --- | --- | | Qwen3.5-0.8B | 0.8B | 0.8B | Dense | 256K | Apache 2.0 | Edge, CPU inference, on-device classification | | Qwen3.5-2B | 2B | 2B | Dense | 256K | Apache 2.0 | Mobile, embedded agents, sub-2GB GPU | | Qwen3.5-4B | 4B | 4B | Dense | 256K | Apache 2.0 | Local coding agent, 6-8GB VRAM | | Qwen3.5-9B | 9B | 9B | Dense | 256K | Apache 2.0 | Reasoning on 12-16GB VRAM laptops | | Qwen3.5-27B | 27B | 27B | Dense | 256K | Apache 2.0 | Single-GPU prod inference | | Qwen3.5-35B-A3B | 35B | 3B | MoE + DeltaNet | 256K | Apache 2.0 | Sweet spot for cost-per-quality | | Qwen3.5-122B-A10B | 122B | 10B | MoE + DeltaNet | 256K | Apache 2.0 | 2x H100 production deployments | | Qwen3.5-397B-A17B | 397B | 17B | MoE + DeltaNet | 256K | Apache 2.0 | Flagship reasoning, agentic workflows | | Qwen3.5-Omni-Plus | ~100B* | — | Thinker-Talker MoE | 256K | Hosted (DashScope) | Multimodal: text, image, audio, video | | Qwen3.5-Plus / Max | — | — | Hosted | 256K-1M | Hosted (DashScope) | Closed-tier production tier | **Omni-Plus active-parameter count has not been published; the unified Thinker-Talker MoE serves text, image, audio, and video through one backbone.* The MoE tiers (35B-A3B, 122B-A10B, 397B-A17B) follow the now-standard pattern: a large total parameter pool with sparse routing so only a fraction of weights activate per token. The naming convention is `{total}-A{active}`; 35B-A3B activates 3B parameters per forward pass out of 35B total, which is what makes it run faster and cheaper than a dense 9B while scoring closer to a dense 35B on quality. If you want a hands-on tour of the smallest tier, our walkthrough on [running and benchmarking Qwen3.5-0.8B](https://codersera.com/blog/run-and-benchmark-qwen35-08b/) covers latency, RAM footprint, and where the 0.8B actually wins in production. ## Architecture: the hybrid that changes the cost curve The single biggest reason Qwen 3.5 is cheaper to serve than its competitors is that most of its layers are not standard softmax attention. Qwen 3.5 interleaves two layer types: full attention with grouped-query attention (GQA) and RoPE, and linear attention layers built on Gated DeltaNet. By default, every fourth layer is full attention. The other three are linear. The trade-off matters. Full softmax attention is O(n²) in sequence length and requires a KV cache that grows linearly with context. Gated DeltaNet is O(n) and does not grow the KV cache at all - it maintains a fixed-size state vector that gets updated with a delta rule plus exponential gating. Stack three DeltaNet layers per softmax layer and you get most of the recall capability of a full-attention model with a fraction of the inference memory. That is why Qwen 3.5 can offer a native 256K context window on a 9B model that fits in 12GB of VRAM. A pure-softmax model with the same context would need several times more memory just for the KV cache. On top of this hybrid attention, the larger tiers add sparse Mixture-of-Experts. The 397B-A17B model has 397B total weights but only routes 17B of them per token. Combined with the linear-attention layers, this is what produces Alibaba's claim of 8-19x faster decoding than the prior Qwen3-Max generation at roughly 60% lower cost. ## Benchmarks: where Qwen 3.5 actually lands The flagship Qwen3.5-397B-A17B competes with frontier closed models on reasoning and agentic benchmarks. The mid-tier 35B-A3B and 27B compete with the best 30-70B models. And the small tier - 4B and 9B - are punching above their weight class because of the unified vision-language training. | Benchmark | Qwen3.5-397B-A17B | DeepSeek V4-Pro | Llama 4 Maverick | Gemma 4 31B | | --- | --- | --- | --- | --- | | GPQA Diamond | 88.4 | 87.9 | 78.2 | 83.6 | | AIME 2026 | 91.3 | 99.4 | 72.0 | 89.2 | | LiveCodeBench v6 | 83.6 | ~85 | 62.4 | 71.0 | | SWE-bench Verified | ~78 (flagship), 73.4 (35B-A3B) | 83.7 | ~55 | ~58 | | MMLU-Pro | 88.0 | 92.8 | 80.5 | 84.2 | | Tau2-Bench (agents) | 86.7 | ~80 | — | ~70 | | MMMU (multimodal) | 85.0 | — | 73.4 | 76.8 | | ArtificialAnalysis Intelligence Index v4.0 | 45 | ~48 | ~35 | ~40 | Some takeaways that don't show in the headline numbers: - **DeepSeek V4 still leads on raw coding** (SWE-bench, AIME 2026 math) because it was trained explicitly for those workloads with the Manifold-Constrained Hyper-Connections architecture. If your use case is whole-codebase refactoring, DeepSeek V4 is the right pick. Our [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) covers that side in depth. - **Qwen 3.5 leads on agentic workflows**. Tau2-Bench at 86.7 is second only to Claude Opus 4.6 (91.6). The Gated DeltaNet memory persistence is what shows up here - long multi-step tool use with consistent state is exactly what linear attention with delta-rule updates is good at. - **The small tier is the surprise**. The 9B model scoring 81.7 on GPQA Diamond is genuinely new. As of early 2026 there was no other model under 30B that broke 80 on Diamond. ArtificialAnalysis confirms the 9B at index 32 and the 4B at 27, both step changes over the Qwen 3 generation. For a head-to-head with the next-most-popular open-weight family at a comparable size, our [Gemma 3 vs Qwen 3 comparison](https://codersera.com/blog/gemma-3-vs-qwen-3-in-depth-comparison-of-two-leading-open-source-llms/) remains a useful baseline; the 3.5 generation widens Qwen's lead on multilingual and long-context, narrows the gap on math, and stays roughly level on general reasoning. ## Pricing: DashScope, DeepInfra, Together, Fireworks Qwen 3.5 is available on essentially every meaningful inference provider. The headline numbers vary by 2-3x depending on host, with DeepInfra typically the cheapest and DashScope the most feature-complete. | Model | DeepInfra (in / out per 1M) | Together (blended) | DashScope | Notes | | --- | --- | --- | --- | --- | | Qwen3.5-0.8B | $0.01 / $0.05 | — | tiered | DeepInfra is the only major host | | Qwen3.5-2B | $0.02 / $0.10 | — | tiered | | | Qwen3.5-9B | ~$0.04 / $0.18 | $0.11/M blended | tiered | Together adds OpenAI-compat layer | | Qwen3.5-35B-A3B | $0.10 / $0.45 | ~$0.20 blended | tiered | The cost-per-quality sweet spot | | Qwen3.5-122B-A10B | $0.30 / $1.50 | ~$0.60 blended | tiered | | | Qwen3.5-397B-A17B | $0.54 / $3.40 | ~$1.20 blended | tiered | Roughly 1/10th of GPT-5.2 pricing | | Qwen3.5-Omni-Plus | — | — | Hosted-only | Audio + video tokens billed separately | DashScope's pricing is tiered by request size rather than flat per-token. Small requests (under 32K tokens) get the lowest rate; long-context calls cost progressively more. This penalizes naive RAG pipelines that stuff context, but rewards well-designed prompts. The DashScope international endpoint is at `dashscope-intl.aliyuncs.com`; the China-mainland endpoint is at `dashscope.aliyuncs.com`. They are not interchangeable - keys are scoped to one region. For teams running real workloads, the practical cost story is: a 50,000-document daily classification job that costs roughly $4,000/month on a frontier closed model lands at around $200/month on Qwen3.5-35B-A3B via DeepInfra with comparable accuracy. That is the cost-per-quality story driving Qwen adoption in 2026. ## Self-hosting: hardware requirements One of the points of Apache 2.0 weights is that you don't have to use a hosted provider at all. Qwen 3.5 is friendly to self-hosting because the linear-attention layers keep the KV cache small. | Model | FP16 / BF16 | Q4_K_M (GGUF) | Realistic GPU | | --- | --- | --- | --- | | 0.8B | ~2 GB | ~0.6 GB | Any laptop, CPU-only OK | | 2B | ~5 GB | ~1.5 GB | RTX 3060 / M1 | | 4B | ~9 GB | ~3 GB | RTX 3060 12GB / M2 | | 9B | ~18 GB | ~5.5 GB | RTX 4060 Ti 16GB / M2 Pro | | 27B | ~54 GB | ~15 GB | RTX 4090 24GB (Q4) / 2x 3090 | | 35B-A3B | ~70 GB | ~22 GB | RTX 5090 32GB / Mac Studio 64GB | | 122B-A10B | ~245 GB FP8: ~123 GB | ~70 GB | 2x H100 80GB FP8 | | 397B-A17B | ~795 GB FP8: ~397 GB | ~220 GB | 4-8x H100 / 2-4x H200 | Three tools cover almost all self-hosting scenarios: - **Ollama** for the simplest path. `ollama pull qwen3.5:9b` and you are running. Best for laptops and dev boxes. - **llama.cpp** for maximum throughput on CPU or Apple Silicon, and for GGUF-quantized MoE deployments where the experts can be offloaded selectively. Our [CPU-only setup guide for the 0.8B with Ollama](https://codersera.com/blog/run-qwen35-08b-with-openclaw-ollama-on-cpu-locally/) shows what's possible without any GPU at all. - **vLLM** for production inference servers with continuous batching, paged attention, and prefix caching. Native support for Qwen 3.5's hybrid architecture landed in vLLM 0.7+. ## Qwen 3.5 as a coding agent (with Claude Code) The most underappreciated angle on Qwen 3.5 is that it speaks the Anthropic Messages API. Alibaba ships a Claude-Code-compatible endpoint at `https://dashscope-intl.aliyuncs.com/apps/anthropic`, which means you can point Claude Code, Cline, or any Anthropic-SDK client at Qwen and run agentic coding workflows without writing any glue. The set-up is short. Set `ANTHROPIC_BASE_URL` to the DashScope Anthropic-compat URL, set `ANTHROPIC_API_KEY` to your DashScope key, and pick a model name like `qwen3.5-coder-plus`. Claude Code starts a session against Qwen and you get the same TUI, tool-use, and file-edit primitives - just with Qwen doing the thinking. For local-only setups, the same trick works against an OpenAI-compatible endpoint served by llama.cpp or vLLM. Our walkthrough on [running Qwen 3.5 with Claude Code as a free local coding agent](https://codersera.com/blog/run-install-and-benchmark-qwen35-claude-code-free-local-ai-coding-agent/) covers both the API-compat path and the fully-offline path with a llama.cpp endpoint, including the prompt-template gotchas that come up when Claude Code's tool-call format meets Qwen's chat template. The practical answer to "is Qwen 3.5 good enough for agentic coding?" is: at the 35B-A3B tier and above, yes for most everyday work; at the 4B-9B tier, yes for tightly-scoped tasks like single-file refactors, test generation, and codemod work where you don't need deep cross-file reasoning. ## Qwen3.5-Omni-Plus and the VL story The Omni-Plus variant is Alibaba's answer to GPT-4o-realtime and Gemini 3.1 Pro: a single model that natively handles text, image, audio, and video, with streaming speech generation through a Thinker-Talker architecture. The Thinker is a Hybrid-Attention MoE that processes all modalities through a unified backbone with a vision encoder for images and an audio tokenizer for sound. The Talker generates contextual speech in a streaming pipeline for real-time interaction. Notable specs: 256K context that can hold over 10 hours of continuous audio or 400 seconds of 720p video at 1 fps with audio; 113 supported languages and dialects, including 39 Chinese dialects; and SOTA results across 215 audio and audio-visual benchmarks per Alibaba's own evaluation. The Realtime API supports semantic interruption detection, native tool use, and voice cloning. For a head-to-head with the closed multimodal field, our [Omni-Plus vs GPT-4o vs Gemini 3.1 Pro comparison](https://codersera.com/blog/qwen3-5-omni-plus-vs-gpt-4o-vs-gemini-3-1-pro-benchmarks-pricing-and-use-cases/) walks through the cases where Omni-Plus wins (Chinese audio understanding, long-form video analysis, cost-per-minute of audio) and where it still trails (English real-time conversation latency, tool-use chaining over voice). If you specifically need a vision-language deployment without the audio pipeline, the older Qwen3-VL line is still in active use; our [Qwen3-VL-30B-A3B-Thinking deployment guide](https://codersera.com/blog/qwen3-vl-30b-a3b-thinking-complete-2025-deployment-guide/) remains accurate for teams that want the document-understanding and OCR capability without paying for the Omni stack. ## Known limitations The marketing rounds usually skip these. They matter. - **Plus and Max tiers are not open-weight.** Qwen3.5-Plus and the implied Qwen3.5-Max sit on DashScope only. If your strategy assumes open weights, the open ceiling is 397B-A17B, not Max. - **Hybrid attention has rough edges in long-context recall.** The 75/25 DeltaNet/softmax mix is excellent on average, but on adversarial needle-in-a-haystack tests at 128K+ it underperforms pure-softmax models of comparable size by a few points. Production RAG pipelines that depend on exact verbatim retrieval over very long contexts should benchmark on real data first. - **Tool-use formatting is finicky outside DashScope.** When you self-host, the exact chat template and the way you serialize tool calls matters. Mismatches between Qwen's expected XML-style tool-call format and Claude Code's expected schema have caused production-blocking bugs that took weeks to diagnose. Pin your inference engine version. - **The fine-tuning ecosystem is still maturing.** Llama has years of LoRA recipes, axolotl configs, and proven SFT corpora. Qwen 3.5 has fewer of those because it's recent. Expect to do more of your own scaffolding for training. - **Omni-Plus's non-English real-time speech still ships rough edges** in Indic languages, Latin American Spanish, and several African languages despite the headline 113-language count. - **SWE-bench Verified at the flagship tier still trails DeepSeek V4.** If raw coding benchmark numbers drive your selection, DeepSeek V4 is the sharper pick for that single dimension. - **Data residency and governance.** DashScope endpoints route to Alibaba Cloud regions in Singapore, Hong Kong, or mainland China. Regulated industries with EU or US-only data residency requirements should self-host or use a Western inference provider rather than DashScope directly. ## FAQ ### Is Qwen 3.5 free to use commercially? Yes for the open-weight tiers (0.8B through 397B-A17B), under Apache 2.0 - no MAU limit, no acceptable-use restrictions, no notice-of-attribution gates. The Plus and Max hosted tiers are paid and governed by DashScope's commercial terms. ### What is the difference between Qwen 3.5 and Qwen 3? Qwen 3.5 unifies text and vision-language into a single backbone (Qwen 3 had separate VL models), introduces the hybrid Gated DeltaNet + softmax architecture, extends native context to 256K, and ships an MoE flagship at 397B total / 17B active. Small-tier intelligence index gains are 9-15 points over the Qwen 3 equivalents. ### How does Qwen 3.5 compare to DeepSeek V4? DeepSeek V4 leads on raw coding and pure-math benchmarks. Qwen 3.5 leads on agentic tasks, multilingual workloads, and the small-tier (under 10B) intelligence frontier. Qwen also ships a wider family with smaller open-weight options. ### Can I run Qwen 3.5 on a laptop? Yes. The 9B model at Q4 quantization needs about 5.5 GB of VRAM. A 16 GB MacBook Pro or any 8 GB-VRAM discrete GPU runs it comfortably with room for context. ### Does Qwen 3.5 support function calling? Yes. All instruct-tuned variants from 4B upward support function calling and tool use. The 397B-A17B and 35B-A3B are the strongest at multi-turn agentic tool sequences. ### Can Qwen 3.5 replace Claude Code's underlying model? Yes. Claude Code accepts an `ANTHROPIC_BASE_URL` override, and DashScope ships an Anthropic-Messages-compatible endpoint. You can also point Claude Code at a local llama.cpp or vLLM endpoint serving Qwen via an OpenAI-compatible shim. ### Is Qwen 3.5 multilingual? Yes - 201 languages on the text models. Omni-Plus adds 39 Chinese dialects and roughly 74 spoken-language coverage on the audio side. ### What is the context window? 256K tokens native across the entire 3.5 family. The hybrid linear attention is what makes that affordable. ### Why does Qwen call its big model "397B-A17B" instead of just 397B? The "A17B" denotes active parameters - the count that actually fires per token. Out of 397B total weights, MoE routing activates 17B. Inference cost scales with active parameters, not total, which is why a 397B model can be served at a fraction of a 397B-dense-model price. ### Should I use the Plus/Max hosted tier or the open-weight 397B-A17B? If you need the absolute best Alibaba can ship and are happy with a hosted model, use Plus/Max. If you need open weights, on-prem deployment, fine-tuning rights, or data residency control, use the 397B-A17B open-weight model. ### Was Qwen 3.5 Max ever released? Not as a GA model. **Qwen3.5-Max-Preview** debuted on LM Arena on March 25, 2026 (closed weights, no public API, score ~1,464; #6 globally at the time), and Alibaba then skipped a 3.5 Max GA — moving straight to **Qwen3.6-Max-Preview** (April 20, 2026, first fully closed Qwen Max) and then **Qwen3.7-Max**, officially announced at the Apsara Summit on May 20, 2026 (see our [Qwen 3.7 release tracker](https://codersera.com/blog/qwen-3-7-release-date-whats-new-2026/) and the [3.7 vs 3.6 comparison](https://codersera.com/blog/qwen-3-7-vs-qwen-3-6-2026/)). If a search for “Qwen 3.5 Max” brought you here, the practical answer in May 2026 is one of two: (a) try **Qwen3.7-Max-Preview** free on chat.qwen.ai for frontier closed-weights reasoning, or (b) self-host the open-weight **Qwen3.5-397B-A17B** MoE on Hugging Face under Apache 2.0 if you need on-prem or fine-tuning rights. The hosted “Max” SKU on DashScope today is the older `qwen3-max` snapshot, not a 3.5 Max. ### How much does it cost to run Qwen 3.5 in production? Hosted: roughly $0.10-$3.40 per 1M output tokens depending on tier. Self-hosted on a 2x H100 box for the 122B-A10B: roughly $4,000/month bare-metal, which becomes cost-effective above approximately 200M tokens/day of usage. ### Does Qwen 3.5 have a "thinking" or reasoning mode? Yes. Most tiers ship Instruct and Thinking variants. The Thinking variants emit chain-of-thought traces and score higher on reasoning benchmarks at the cost of longer generation latency. ### Is Qwen 3.5 safe to deploy in regulated environments? The weights are open and inspectable - that's a positive for compliance. But DashScope hosted endpoints route through Alibaba Cloud regions, which often does not satisfy EU / US-only data residency requirements. Self-hosting or Western providers like DeepInfra, Together, and Fireworks are the standard path for regulated workloads. ### What's the practical cost difference vs GPT-5.2 or Claude Opus 4.6? Order-of-magnitude. The Qwen3.5-397B-A17B at $0.54 / $3.40 per 1M tokens is roughly 1/10th to 1/15th the price of the leading frontier closed models for comparable agentic and reasoning quality. ## Next steps Picking and deploying an open-weight model is the easy part. The hard part is the engineering around it: data pipelines, fine-tuning recipes, agent scaffolding, eval harnesses, and the production observability to know when the model regresses on your real workload. This is the kind of work that bottlenecks teams for months when done part-time alongside everything else. If you're scaling Qwen 3.5, DeepSeek V4, or any open-weight model into production and you need senior engineers who already know the stack - vLLM, llama.cpp, LoRA fine-tuning, eval design, and the agent-tooling glue - [Hire a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer). Codersera-vetted developers are remote-ready, technically screened, and available on a risk-free trial so you can validate fit before committing. =============================================================== [6/20] Gemma 4: The Complete Developer Guide (2026) URL: https://codersera.com/blog/gemma-4-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== *Last updated: May 1, 2026* Gemma 4 is the most consequential open-weight model release of the year so far, and not just because of the benchmarks. Google shipped four model sizes, native multimodality, a 256K context window on the larger variants, and — for the first time in the Gemma line — a clean Apache 2.0 license. For engineering teams that have been waiting for an open-weight model good enough to actually replace a frontier API for a meaningful chunk of their workload, this is the first credible candidate from Google. This guide is the long version: what the family looks like, what the architecture actually does, what the benchmark numbers mean in practice, how it stacks up against Llama 4, Qwen 3.5, DeepSeek V4 Flash, and its own predecessors, where to host it, and where it falls short. If you are evaluating Gemma 4 for production, this is the document to send your team. ## TL;DR - **Released:** April 2, 2026, by Google DeepMind. - **Family:** four sizes — Gemma 4 E2B (~2.3B effective), E4B (~4.5B effective), 26B A4B (Mixture-of-Experts, 4B active), and 31B dense. - **License:** Apache 2.0. This is new. Earlier Gemma generations shipped under the custom *Gemma Terms of Use*, which had usage carve-outs that made enterprise legal review painful. Gemma 4 dropped that. - **Context window:** 128K tokens on E2B/E4B; 256K on 26B A4B and 31B. - **Multimodal:** all sizes accept text + image; E2B and E4B also accept audio. Output is text-only. - **Strong points:** reasoning, math (AIME 2026 ~89%), code generation (LiveCodeBench v6 ~80%), long-context recall, and on-device deployment via MediaPipe / LiteRT. - **Weak points:** trails Qwen 3.5 27B on SWE-bench Verified, no native speech output, and Gemma is not Gemini — fine-tuning, weights, and serving are now your problem. ## What Gemma 4 Is, And How It Differs From Gemini Gemma is Google's open-weight model family. Gemini is Google's closed, hosted, frontier model family. They share research lineage — Gemma 4 is described by Google as "built from Gemini 3 research" — but the deployment story is different. With Gemini you call an API, you pay per token, you do not get the weights, and you cannot fine-tune the underlying parameters (you get adapters at best). With Gemma 4 you download the weights from Hugging Face, Kaggle, or Ollama, you run them on your own hardware (or a cloud GPU you rent), you fine-tune fully, and your unit economics are GPU hours and electricity rather than per-token API spend. The practical implication: Gemma 4 is the model you reach for when you need on-device inference, when you need to fine-tune on private data, when your token volume makes a hosted API uneconomical, or when you need an air-gapped deployment. Gemini is the model you reach for when you want zero-ops frontier intelligence and you are happy to pay for it. For a deeper feature-level walkthrough, see our companion piece [Google Gemma 4 review: benchmarks, features, and how to run it locally](https://codersera.com/blog/google-gemma-4-review-benchmarks-features-run-locally/). ## The Gemma 4 Family Four sizes, two architectural patterns (dense and MoE), and a clear split between edge and server tiers. | Variant | Architecture | Total / Active params | Context | Modalities in | Primary target | | --- | --- | --- | --- | --- | --- | | Gemma 4 E2B | Dense | ~2.3B effective | 128K | Text, image, audio | Phones, IoT, low-power laptops | | Gemma 4 E4B | Dense | ~4.5B effective | 128K | Text, image, audio | High-end phones, edge servers, Raspberry Pi-class | | Gemma 4 26B A4B | Mixture-of-Experts | 26B total / ~4B active per token | 256K | Text, image | Single high-end GPU server, cost-sensitive throughput | | Gemma 4 31B | Dense | 30.7B | 256K | Text, image | Quality-first server inference, fine-tuning | The "E" in E2B/E4B is for *edge*, not *experts*. These are dense models built for on-device. The 26B A4B is the MoE: 4 billion parameters fire on any given forward pass, so latency and cost behave like a 4B model, while quality benefits from the full 26B parameter pool. The 31B is the no-tricks dense model — slower than the MoE, but typically the highest-quality answer when you need the best response per query rather than the best response per dollar. If you are deciding which one to actually pull, our breakdown [Gemma 4 vs Gemma 3 vs Gemma 3n: which makes sense in 2026](https://codersera.com/blog/gemma-4-vs-gemma-3-vs-gemma-3n-which-model-makes-the-most-sense-in-2026/) walks the decision tree per workload. ## Architecture, Context Window, And Tokenizer Gemma 4 keeps the decoder-only transformer skeleton that has defined the family but tightens almost every component. Highlights worth knowing: - **Hybrid attention.** Gemma 4 interleaves local sliding-window attention with full global attention, with the final layer always global. Smaller dense models use 512-token sliding windows; larger models use 1024. This is what makes the 256K context feasible without VRAM blowing up linearly. - **RULER long-context recall.** On RULER at 128K, Gemma 3 scored 13.5%. Gemma 4 scores 66.4% on the same test. The context window is not just nominal — it actually retrieves at depth. - **Vocabulary.** 262,144-token vocabulary, BPE with byte fallback. Strong multilingual coverage — 140+ languages. - **Vision tokens.** Variable visual budget (70, 140, 280, 560, or 1120 tokens per image), so you trade quality against context spend. - **Audio (E2B/E4B only).** Native speech recognition and audio understanding, no separate ASR layer required for many use cases. - **Reasoning mode.** Gemma 4 can produce 4,000+ tokens of explicit reasoning before committing to an answer, plus native function-calling and structured JSON output. The MoE in 26B A4B is the architectural story to internalise: it lets a single A100 80GB or two consumer GPUs serve a model that punches well above 4B in quality terms, at roughly 4B in cost terms. That is the new dominant design point for the open-weight server tier in 2026. ## License: Apache 2.0, Finally Read this section carefully if you have ever had Legal kill a Gemma rollout. Earlier Gemma releases shipped under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms), a custom license. It was more permissive than Llama 2's, but it included a *Prohibited Use Policy* with clauses around harm to minors, attacks on critical infrastructure, generation of CSAM, and other broad carve-outs. The clauses were defensible in spirit, but enterprise legal teams routinely flagged the language as ambiguous and asked for indemnification or scope-limiting before signing off. That friction kept Gemma out of plenty of production stacks. **Gemma 4 ships under Apache 2.0.** No custom restrictions, no usage carve-outs, no monthly active user thresholds (the way the Llama 4 Community License has). Apache 2.0 explicitly grants commercial use, modification, redistribution, and distribution of derivative works, including derivative weights. There is one obvious constraint that still applies: Apache 2.0 does not grant trademark rights, so you cannot ship a product called "Gemma" or imply Google endorsement. This is materially less restrictive than the previous Gemma Terms of Use, and noticeably less restrictive than Llama 4's Community License (which is free for organisations under 700M monthly active users but adds compliance language). For most engineering teams, this is the change that turns Gemma from "interesting" into "approvable." Two caveats worth being honest about. First, Apache 2.0 governs the *weights*; it does not give you the training data or the training pipeline. Gemma 4 is open-weight, not open-source in the strict OSI sense applied to data. Second, Google can still publish acceptable-use guidelines separately; nothing about Apache 2.0 prevents that. Today, the license file in the repo is the controlling document — and that document is Apache 2.0. ## Benchmarks That Actually Matter Headline numbers for Gemma 4 31B (instruction-tuned), pulled from Google's model card and independent reproductions on the LM Studio and Hugging Face threads: | Benchmark | Gemma 4 31B | Gemma 3 27B | Llama 4 Scout (109B) | Qwen 3.5 27B | DeepSeek V4 Flash | | --- | --- | --- | --- | --- | --- | | MMLU-Pro | 85.2 | ~67 | ~78 | 86.1 | ~84 | | GPQA Diamond | 84.3 | 42.4 | ~70 | 85.5 | ~80 | | LiveCodeBench v6 | 80.0 | 29.1 | ~55 | ~78 | ~74 | | SWE-bench Verified | ~63 | ~22 | ~48 | 72.4 | ~64 | | AIME 2026 (math) | 89.2 | 20.8 | ~55 | ~85 | ~82 | | Codeforces ELO | 2,150 | 110 | ~1,500 | ~1,950 | ~1,800 | Approximate values for non-Gemma rows are pulled from each project's own card or the Artificial Analysis index; treat them as directional. The story they tell is consistent: - Gemma 4 31B is in the same neighbourhood as Qwen 3.5 27B on knowledge and reasoning. They trade leadership benchmark by benchmark. - Gemma 4 has the upper hand on math and competitive programming. - **Qwen 3.5 27B still wins SWE-bench Verified** — the benchmark that most closely tracks "can this model close a real GitHub issue." If your primary use case is autonomous code editing on real repos, evaluate Qwen 3.5 alongside Gemma 4 before you commit. - Gemma 4's gain over Gemma 3 is enormous — multiple benchmarks improved 3–20×. Most teams running Gemma 3 in production should plan a migration window. For the pairwise drilldowns: [Gemma 4 vs Llama 4 for local deployment](https://codersera.com/blog/gemma-4-vs-llama-4-local-deployment-2026/), [Gemma 4 vs Gemma 3: what changed and should you switch](https://codersera.com/blog/gemma-4-vs-gemma-3-what-changed-and-should-you-switch/), [Gemma 4n vs Gemma 4](https://codersera.com/blog/gemma-4n-vs-gemma-4/), and [our DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). ## Where To Run Gemma 4 You have three deployment surfaces: hosted, self-hosted server, and on-device. ### Hosted If you want zero ops, the model is a one-line call away on several providers: - **Vertex AI (Model Garden).** First-party. You can fine-tune on Vertex AI Training Clusters and serve through Model Garden endpoints. Pay for compute time on the underlying accelerator (A2/G2 family or TPUs). - **OpenRouter.** Aggregates 11+ providers for the 26B A4B model at roughly $0.06 per million input tokens and $0.33 per million output. Useful for prototyping and price-sensitive batch work. - **Together AI, Fireworks, Groq, DeepInfra, Hugging Face Inference.** All have Gemma 4 endpoints. Pricing varies but the open-weight competitive market keeps it low. - **Cloud Run with GPU.** Google's serverless GPU runtime can host Gemma 4 with scale-to-zero, which is attractive for spiky workloads. ### Self-hosted server vLLM is the production default. It supports Gemma 4 on NVIDIA, AMD, and Google Cloud TPUs from day one. Approximate hardware floors: | Variant | Quant / format | VRAM floor | Notes | | --- | --- | --- | --- | | 26B A4B | AWQ INT4 | ~15 GB | RTX 4090 24 GB with KV-cache headroom | | 26B A4B | GGUF Q4_K_M | ~16 GB | llama.cpp / Ollama dev box | | 26B A4B | FP16 | ~52 GB | A100 80GB or H100; serves at full quality | | 31B dense | FP16 | ~62 GB | A100 80GB or H100 single-GPU | | 31B dense | INT4 | ~18 GB | RTX 4090 / 5090 — viable for single-user inference | Ollama covers the local-laptop use case for E2B, E4B, and the quantised 26B/31B. MLX with Metal acceleration runs all variants on Apple Silicon — an M3 Max or M4 Pro with 32–64 GB unified memory will run the 26B A4B comfortably. AMD has day-zero Gemma 4 support across ROCm and the Ryzen AI stack. NVIDIA NIM, NeMo, LM Studio, Unsloth, SGLang, and LiteRT-LM all have first-class support. For a step-by-step Ollama setup, see [How to run Gemma 4 with Ollama](https://codersera.com/blog/how-to-run-gemma-4-with-ollama-setup-guide/), and for a hardware-centric walkthrough, [Run Gemma 4 on your PC and devices locally](https://codersera.com/blog/run-gemma-4-on-your-pc-and-devices-locally/). ### On-device with MediaPipe and LiteRT The E2B and E4B variants are explicitly designed for phones and edge devices. The deployment stack is MediaPipe's LLM Inference API on top of LiteRT, which handles model loading, memory, and hardware acceleration (GPU or NPU) automatically. Approximate footprints: - E2B Q4_K_M: ~1.3 GB on disk, 2–3 GB RAM at runtime. - E4B Q4_K_M: ~2.5 GB on disk, 4–5 GB RAM at runtime. This is the path for "AI features that work without a network round-trip" — voice agents on Android, in-browser RAG over a user's local documents, and offline coding helpers. With audio input native to E2B/E4B, you can ship a meaningful voice-to-text-to-action loop without bundling a separate ASR model. ## When To Choose Gemma 4 Over Alternatives Reach for Gemma 4 when: - **You need an Apache 2.0 model.** If Legal balked at Gemma 3's terms or Llama's Community License MAU clause, Gemma 4 is the cleanest option in this size class. - **You need on-device multimodality.** The audio-capable E2B/E4B variants are the strongest open-weight option for phones today. - **Long context matters.** 256K with credible RULER recall is competitive with hosted frontier models. - **Math, agentic reasoning, or competitive programming dominate your workload.** Gemma 4 31B's AIME and Codeforces numbers are exceptional for an open-weight model in this size band. Choose something else when: - **Your workload is autonomous repo editing.** Qwen 3.5 27B's SWE-bench Verified lead is real. Pilot both before committing. - **You need streaming voice output.** Gemma 4 has audio in but not out. Qwen 3.5-Omni handles real-time speech generation. - **You need a frontier model.** If quality is the only metric, hosted Gemini 3 Pro or DeepSeek V4 Pro will outperform Gemma 4 31B on most benchmarks. - **Cost-per-token at huge scale.** DeepSeek V4 Flash hosted is cheap enough that for many workloads the spend math beats running your own GPUs. ## Known Issues And License Caveats - **SWE-bench Verified is not the strong suit.** Real GitHub issue resolution still trails Qwen 3.5 27B by a meaningful margin. - **No native audio output.** If you want a voice agent that talks back, you bolt on a separate TTS layer. - **26B A4B throughput surprise.** Despite only 4B active parameters, community benchmarks on consumer GPUs show ~11 tok/s on an RTX 4090 — slower than a comparable dense 4B model. The MoE routing overhead is real on consumer hardware. On A100/H100 the gap closes. - **Apache 2.0 ≠ open-source training data.** The weights are open and commercially usable; the training corpus is not. If your compliance posture requires reproducibility from data, Gemma 4 does not satisfy that. - **Trademark.** You cannot brand your product as "Gemma" or use Google trademarks. Apache 2.0 explicitly excludes trademark grants. - **Vision token budget tradeoff.** The 70/140/280/560/1120 visual budgets are real — undersized budgets degrade OCR and chart reading noticeably. Pick deliberately. - **better-sqlite3-style native dependencies.** If you self-host with vLLM behind a Node service, watch out for prebuilt-binary fetch issues on locked-down installs; the failure mode is silent at install time and loud at runtime. - **Tokenizer drift from Gemma 3.** The 262K vocabulary is not directly weight-compatible with Gemma 3 fine-tunes. Plan a re-finetune, do not try to port adapters. ## FAQ ### Is Gemma 4 actually open-source? It is open-weight under Apache 2.0. The weights, model card, and inference code are open and commercially usable. The training data and full pipeline are not released. By the OSI's strict definition, that is open-weight, not open-source — but for most commercial deployment purposes, Apache 2.0 is the cleanest license you will see in this size class. ### Is the Gemma 4 license really Apache 2.0? Yes. This is the change from earlier Gemma versions, which used the custom Gemma Terms of Use with usage carve-outs. Gemma 4's repository ships the standard Apache 2.0 license file. Anyone telling you Gemma 4 has restrictive terms is describing the previous generation. ### What is the difference between Gemma 4 and Gemini? Gemma 4 is open-weight and self-hostable; Gemini is a closed, hosted, frontier model. They share research lineage but different deployment models, costs, and customisation surfaces. ### Which Gemma 4 model should I pick? E2B for phones and tight memory budgets, E4B for high-end edge and small servers, 26B A4B for cost-efficient single-GPU server inference, 31B dense for highest-quality answers when you do not care about throughput. ### What hardware do I need to run Gemma 4 31B? FP16 needs roughly 62 GB VRAM — A100 80GB or H100. INT4 quantised drops that to about 18 GB, fitting an RTX 4090 or 5090 for single-user inference. ### Does Gemma 4 support function calling? Yes. Native function calling, structured JSON output, and system instructions are first-class. ### How does Gemma 4 compare to Llama 4? Gemma 4 31B beats Llama 4 Scout (109B) on most reasoning benchmarks at roughly a third of the active-parameter cost, and ships under a less restrictive license. ### Is Gemma 4 better than Qwen 3.5? It depends on the workload. Gemma 4 wins on math and competitive programming; Qwen 3.5 27B wins on MMLU-Pro, GPQA Diamond, and SWE-bench Verified. Both are Apache 2.0. Pilot both. ### Is Gemma 4 multimodal? All variants accept text and image. E2B and E4B also accept audio. Output is text-only on every variant. ### What is the context window? 128K tokens on E2B/E4B; 256K on 26B A4B and 31B. RULER long-context recall at 128K is roughly 66.4% — a 5× improvement over Gemma 3. ### Can Gemma 4 run on a phone? Yes. E2B and E4B are designed for it. MediaPipe's LLM Inference API and LiteRT handle on-device inference with NPU and GPU acceleration on Android, and equivalent paths exist on iOS via Core ML / MLX. ### What is Gemma 4n? "Gemma 4n" is the community shorthand for the E2B / E4B edge variants — the on-device tier of the Gemma 4 family. Architecturally they are dense models tuned and quantised for phones and embedded devices. See [Gemma 4n vs Gemma 4](https://codersera.com/blog/gemma-4n-vs-gemma-4/) for the side-by-side. ### Is Gemma 4 safe for commercial production use? Yes, under Apache 2.0, with the standard caveats: respect trademarks, do not redistribute the model under the Gemma name, and follow your own jurisdiction's AI usage law. There are no usage carve-outs, MAU thresholds, or industry restrictions in the license itself. ### Should I migrate from Gemma 3 to Gemma 4? If you are running Gemma 3 in production, yes. The benchmark deltas are large (3–20× on reasoning and code), the license is cleaner, the context window is bigger, and the deployment story is unchanged. Plan a re-finetune — adapter weights will not transfer cleanly. ## Next Steps Picking the right open-weight model is the easy half. The hard half is hiring engineers who can fine-tune it on your data, harden the inference path, and ship it without bricking your unit economics. [Hire a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer) who has actually deployed Gemma-class models on vLLM, MediaPipe, and MLX. Vetted technical fit, remote-ready, and a risk-free trial so you only keep the engineers who deliver. =============================================================== [7/20] Kimi K2.6: The Complete Developer Guide (2026) URL: https://codersera.com/blog/kimi-k2-6-complete-guide-2026/ Last updated: 2026-08-13 =============================================================== **Newer model:** Moonshot has since released [**Kimi K2.7 Code**](https://codersera.com/blog/kimi-k2-7-complete-guide-2026) (June 2026) — a coding-focused 1T-MoE upgrade with ~30% fewer thinking tokens. See our [full Kimi K2.7 guide](https://codersera.com/blog/kimi-k2-7-complete-guide-2026). **Quick answer.** Kimi K2.6 is Moonshot AI's flagship open-weights model, released April 20, 2026. It is a 1T-parameter MoE with 32B active per token, native INT4 quantisation, and a new Agent Swarm primitive that fans out to 300 sub-agents across 4,000 coordinated steps. It scores 54 on the Artificial Analysis Intelligence Index — the highest of any open-weights model — and ties GPT-5.5 on SWE-bench Pro at roughly one-fifth the token cost. *Last updated: May 14, 2026.* Moonshot AI shipped Kimi K2.6 on April 20, 2026 and reset what "open weights" can mean. On the headline benchmark every frontier lab now leans on — Artificial Analysis Intelligence Index — K2.6 scores 54, the highest of any open-weights model and only three points behind Anthropic, Google, and OpenAI's closed flagships. On SWE-bench Pro it ties GPT-5.5. On Code Arena's WebDev leaderboard it sits sixth out of 67 models at 1,529 Elo, ahead of every other open-weights model and within striking distance of Claude Opus 4.7. And it runs at roughly one-fifth the per-token cost of Opus 4.7. The release also introduces a genuinely new primitive: Agent Swarms. K2.6 has been post-trained — not just prompted — to decompose long-horizon work into up to 300 parallel sub-agents and reconcile their results across as many as 4,000 coordinated steps. Moonshot's reference run shows the model sustaining a continuous 12-hour autonomous coding session that made 4,000+ tool calls and raised the throughput of a Qwen 3.5-0.8B inference engine in Zig on a Mac from 15 to 193 tokens per second. This guide is for engineering leaders, founders, and developers deciding where K2.6 fits in a 2026 stack alongside Claude Opus 4.7, GPT-5.5, and DeepSeek V4 Pro. We cover what is genuinely new, the benchmark picture without the hype, the API and self-hosting paths, where K2.6 wins and where it still loses, and the practical decision tree for routing tasks across models. ## Also in this series Also in this series - [**Claude Opus 4.7: Complete Guide (2026)**](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/) — the closed-weights coding leader K2.6 chases on SWE-bench Verified. - [**GPT-5.5: Complete Guide (2026)**](https://codersera.com/blog/gpt-5-5-complete-guide-2026/) — the broad-tool generalist K2.6 ties on SWE-bench Pro. - [**DeepSeek V4: Complete Guide (2026)**](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) — the other open-weights flagship, released four days after K2.6. - [**Open-Source LLMs Landscape 2026**](https://codersera.com/blog/open-source-llms-landscape-2026/) — where K2.6 sits in the wider open-weights field. ## What is Kimi K2.6? Kimi K2.6 is the third K2-class model from Moonshot AI, the Beijing-based lab founded by Yang Zhilin in 2023. The K2 line is Moonshot's open-weights flagship; the closed Kimi consumer product (kimi.com) runs on the same family. K2 launched in August 2025, K2.5 ("K2-Thinking") followed in November 2025, and K2.6 shipped on April 20, 2026 — a nine-month cadence faster than any closed-weights lab. The architecture is a 1-trillion-parameter sparse Mixture-of-Experts with 32 billion active parameters per token. It uses Multi-head Latent Attention (MLA), 384 routed experts plus 1 shared, 8 experts selected per token, 61 transformer layers, 64 attention heads, and SwiGLU activation. The context window is 262,144 tokens. The MoonViT vision encoder (400M params) handles native image and video input. Weights are released under a Modified MIT license that imposes no commercial restriction below ~100M MAU or $20M/month revenue. Why K2.6 matters in one sentence: it is the first open-weights model that competes with the closed frontier on every dimension that matters for agentic coding — long-context stability, tool-call reliability, sub-agent orchestration, and unit economics — without an asterisk. Where K2.5 was "open weights, but with caveats," K2.6 is "open weights, no caveats." ## What's new versus Kimi K2.5? Three changes do most of the work. Everything else is delta on existing capability. **1. Agent Swarms.** K2.5 capped concurrent sub-agents at 100 and coordinated steps at 1,500. K2.6 raises both — 300 sub-agents and 4,000 steps — and, more importantly, post-trains the model to decide on its own when to fan out, how many agents to spawn, and how to reconcile results. The native primitive is what differentiates K2.6 from "K2.5 with a bigger context." On BrowseComp, plain K2.6 scores 83.2%; with swarms enabled, 86.3%. The lift is bigger on naturally parallel work (multi-repo refactors, batch validation, large literature reviews) and smaller on linear tasks. **2. Native INT4 quantisation.** Moonshot used Quantisation-Aware Training during post-training, so the model learned representations compatible with 4-bit weights rather than being compressed afterwards. The practical result: roughly 2x inference speed and 50% less GPU memory versus FP16, with Moonshot claiming negligible quality loss. The INT4 weights on Hugging Face are ~594 GB. **3. Hard benchmark lift across coding and reasoning.** SWE-bench Verified rises from 76.8% (K2.5) to 80.2%. SWE-bench Pro rises from 50.7% to 58.6%. Terminal-Bench 2.0 rises from 50.8% to 66.7%. LiveCodeBench v6 rises from 85.0% to 89.6%. AIME 2026 lands at 96.4%, HMMT 2026 at 92.7%, GPQA-Diamond at 90.5%. Hallucination rate on AA-Omniscience drops from 65% (K2.5) to 39% (K2.6), close to Claude Opus 4.7. Smaller deltas worth knowing before you migrate K2.5 client code: - **New thinking field.** The chat template now exposes `thinking: {type: "enabled" | "disabled", keep: "all"}`. K2.5 client code that does not parse `response.choices[0].message.reasoning` will silently lose the reasoning trace. - **Default temperature is 1.0.** K2.5 defaulted to 0.6. Copying K2.5 prompts forward without re-tuning yields more creative variance in thinking mode. - **Transformers version pin.** Native INT4 inference needs `transformers>=4.57.1, <5.0.0`. Older inference stacks silently fall back to FP16 and OOM on single-node setups. - **Vocabulary 160K.** Up from K2.5's 152K. Modest reduction in tokens-per-request for code and non-English text. - **Vision encoder doubled.** MoonViT scaled from 200M to 400M params. Dense documents, IDE screenshots, and dense data tables are now competitive with Opus 4.7 (MMMU-Pro 79.4%, MathVision-with-python 93.2%). ## How does the Agent Swarm architecture work? Most agent frameworks today — LangGraph, CrewAI, AutoGen — bolt orchestration on top of the model in user space. The model is a black-box generator; the framework owns spawning, scheduling, and reconciliation. K2.6 internalises that primitive. The model is trained to decide when to fan out, how many sub-agents to spawn, what each one does, and how to combine their results. There are two related primitives. **Agent Swarms** let the model spawn up to 300 homogeneous sub-agents for parallelisable work — for example, 200 sub-agents each reading a single file in a large monorepo to localise a regression, then a reconciliation step that proposes a fix. **Claw Groups** extend this to heterogeneous agents with persistent memory: a planner-agent, a researcher-agent, a coder-agent, and a verifier-agent each maintain their own scratchpad across the full session, share intermediate state through a structured memory protocol, and pick up where they left off after a tool failure. What this buys you in practice: - **Quality lift on parallelisable work.** BrowseComp jumps from 83.2% (plain) to 86.3% (with swarms). The lift is bigger on tasks with naturally parallel structure and smaller on linear tasks. - **Latency reduction at fixed quality.** A 20-minute single-agent run can drop to 3–5 minutes when the work decomposes into 20 parallel sub-tasks. The token bill is roughly the same; wall-clock is dramatically lower. - **Cost discipline.** Sub-agents inherit the parent's task budget. They cannot silently spawn an exponential tree of grandchildren. Moonshot's reference scheduler caps each task at 300 active agents and 4,000 total steps; you can lower these in your client. - **Recovery from tool failures.** A failed sub-agent does not halt the parent. The parent receives a structured failure report and decides whether to retry, replan, or proceed without that result. K2.6's 96.6% tool-invocation success rate matters here — the remaining 3.4% is handled by the swarm, not by the human. The honest caveat: swarms shine on parallelisable work and add overhead on inherently sequential work. A bug fix that requires reading file A, then file B, then file C, then writing a patch is not a swarm problem. A bug fix that requires reading 80 files in a monorepo to understand a regression — that is a swarm problem. Disable swarms with an extra-body flag when you know the task is linear. ## What benchmarks does Kimi K2.6 score on? Benchmarks narrow your shortlist; they do not pick a winner. The numbers below are the ones to remember. **Artificial Analysis Intelligence Index (composite, all categories): 54.** Highest of any open-weights model. Ranked #4 overall behind Anthropic, Google, and OpenAI (all 57). Median open-weights model: 30. Xiaomi's MiMo V2.5 Pro recently tied at 54 with weights expected shortly. **SWE-bench Pro: 58.6%.** Tied with GPT-5.5 (57.7%). Ahead of Gemini 3.1 Pro (54.2%) and Claude Opus 4.6 (53.4%). Behind Claude Opus 4.7 (64.3%). **SWE-bench Verified: 80.2%.** Within a tight band of every top-tier model. DeepSeek V4 Pro ties at 80.6%; Opus 4.7 leads at 87.6%. **Code Arena WebDev (Elo, blind pairwise): 1,529.** Sixth of 67 models as of April 26, 2026. Behind Claude Opus 4.7 (1,565), Claude Opus 4.6 (1,548), and Z.ai's open-weights GLM-5.1 (1,534). **Terminal-Bench 2.0 (Terminus-2 harness): 66.7%.** Ahead of GPT-5.4 and Claude Opus 4.6 (both 65.4%). Behind Gemini 3.1 Pro (68.5%) and GPT-5.5 (~82.7%). **LiveCodeBench v6: 89.6%.** Up from K2.5's 85.0%. DeepSeek V4 Pro leads at 93.5%. **BrowseComp (web research): 86.3% in Agent Swarm mode.** Up from K2.5's 78.4%. Plain mode: 83.2%. **DeepSearchQA (F1): 92.5.** Leads GPT-5.4 (78.6). **Humanity's Last Exam (HLE-Full, with tools): 54.0.** Leads every model in the comparison, including GPT-5.4 (52.1), Claude Opus 4.6 (53.0), and Gemini 3.1 Pro (51.4). **AIME 2026: 96.4%. HMMT 2026: 92.7%. GPQA-Diamond: 90.5%.** Highest of any open-weights model on competition math and graduate-level science. **Tool-invocation success: 96.6%.** Highest of any model with public weights. Remaining 3.4% is largely malformed third-party MCP server schemas, not model errors. **Hallucination rate (AA-Omniscience): 39%.** Down from K2.5's 65%, closing on Claude Opus 4.7's ~31%. The honest read: K2.6 is the best model in the world right now for the specific shape of work that is "run unsupervised for hours, fan out into sub-agents, finish a real coding task" — and the best open-weights model on almost every coding benchmark in the suite. Opus 4.7 still wins SWE-bench Verified and graduate-science questions. GPT-5.5 still wins terminal-style breadth. DeepSeek V4 Pro is still cheaper on raw output cost and leads competitive programming. ## How does Kimi K2.6 compare to Claude Opus 4.7, GPT-5.5, and DeepSeek V4 Pro? The 2026 frontier is now a four-way race rather than a three-way one. The way to think about routing: **vs Claude Opus 4.7.** Opus 4.7 wins SWE-bench Verified (87.6% vs 80.2%), MCP-Atlas (~77% vs ~74%), and graduate-level science (GPQA Diamond 94.2% vs 90.5%). K2.6 wins on cost (5–6x cheaper), open weights, agent swarms, web research, multilingual coding, and competitive programming. Pick Opus 4.7 when the bottleneck is "can it ship this single PR." Pick K2.6 when the bottleneck is "can it run unsupervised for hours, coordinate sub-agents, and finish a multi-task." For the deep comparison, see our [Claude Opus 4.7 complete guide](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/). **vs GPT-5.5.** GPT-5.5 wins on the AAII composite (60 vs 54), Terminal-Bench 2.0 (~82.7% vs 66.7%), and time-to-first-token latency (~0.8s vs 3.0s in thinking mode). K2.6 ties on SWE-bench Pro, leads on web research (DeepSearchQA 92.5 vs ~80) and competition math, and is roughly 3x cheaper. Pick GPT-5.5 when terminal/agent breadth dominates. Pick K2.6 when the workload is structured, parallelisable, and cost-sensitive. See the [GPT-5.5 complete guide](https://codersera.com/blog/gpt-5-5-complete-guide-2026/) for the inverse perspective. **vs DeepSeek V4 Pro.** V4 Pro shipped four days after K2.6 (April 24, 2026) and is the closest peer. The two trade wins: K2.6 leads on agentic tooling (HLE-with-tools, BrowseComp with swarms, DeepSearchQA, multilingual SWE), V4 Pro leads on competitive programming (LiveCodeBench 93.5 vs 89.6) and ties on SWE-bench Verified (80.6 vs 80.2). On the GDPval-AA agentic real-world benchmark, V4 Pro currently leads (1554 vs 1484). On raw output cost, V4 Pro is about **4.6x cheaper** ($0.87/M vs K2.6's $4.00/M) — that discount is DeepSeek's standing price, not a promo. From August 16, 2026 V4 Pro moves to $1.98/M output off-peak and $3.96/M at peak, which narrows the gap to ~2x and then roughly parity. The differentiation is concentrated in long-horizon agentic work: K2.6's post-training specifically targets it; V4's does not. For 12+ hour autonomous coding sessions, K2.6 is the clear win; for everything else, it comes down to license, context length (V4 has 1M, K2.6 has 256K), and price-window. See the [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) for the V4-side argument. Kimi K2.6 deep dives - [**Kimi K2.6 vs DeepSeek V4 vs GLM-5.1**](https://codersera.com/blog/kimi-k2-6-vs-deepseek-v4-vs-glm-5-1-2026/) — the three-way open-weights coding verdict, benchmark by benchmark. - [**Kimi K2.6 vs Claude Opus 4.7**](https://codersera.com/blog/kimi-k2-6-vs-claude-opus-4-7/) — open weights against the closed-weights coding leader, head to head. - [**Kimi K2.6 vs GPT-5.5**](https://codersera.com/blog/kimi-k2-6-vs-gpt-5-5/) — agentic open weights versus OpenAI’s flagship generalist. ## How much does Kimi K2.6 cost via API? Moonshot publishes hosted-API pricing at `https://api.moonshot.ai/v1`. The API is OpenAI-compatible, so existing OpenAI SDKs work without modification. Pricing is the same on the .ai and .cn endpoints; OpenRouter and Cloudflare Workers AI re-sell the model at small markups. Headline rates as of May 2026: - **Moonshot direct:** $0.95 / $4.00 per million input/output tokens. Cache reads $0.16/M (83% off input). - **OpenRouter:** $0.74 / $3.50 per million input/output tokens. - **Cloudflare Workers AI, NVIDIA NIM, DeepInfra, GMI Cloud:** blended $1.15–$2.15 per million, depending on provider. The cache discount is the unusual part. Repeated prefixes are cached and billed at $0.16/M on read, applied automatically — no explicit cache-control markers like Anthropic's API requires. The blended rate at a 3:1 input:output ratio works out to roughly $1.71 per million tokens, 5–6x cheaper than Claude Opus 4.7 and ~3x cheaper than GPT-5.5. To make this concrete: take an autonomous coding agent fixing 100 medium-complexity bugs per day, with 50,000 cached context tokens, 5,000 fresh input tokens per task, and 8,000 output tokens per task. - **K2.6 first task (cold cache):** ~$0.085. - **K2.6 subsequent task (warm cache):** ~$0.045. - **K2.6 daily total (100 tasks):** ~$4.50/day, or ~$135/month. - **Same workload on Opus 4.7:** ~$750/month — about 5.5x more. - **Same workload on DeepSeek V4 Pro:** ~$28/month — roughly **5x cheaper than K2.6**. What launched as a 75%-off promo became DeepSeek's standing price on 2026-05-22 ($0.435 / $0.87 per 1M in/out), so the old ~$113/month figure computed against the retired $1.74 / $3.48 list rate no longer applies. - **Same workload on DeepSeek V4 Pro after August 16, 2026:** ~$62/month off-peak and ~$123/month at peak. DeepSeek moves both V4 models to peak/off-peak tiers at 16:00 UTC that day, and every tier bills above the current flat rate — which narrows its lead over K2.6 to roughly 2x off-peak and closes it at peak. [Full breakdown](https://codersera.com/blog/deepseek-v4-price-change-august-2026/). - **Self-hosted K2.6 on 8x H200 at on-demand cloud rates:** ~$580/month at 60% utilisation, before storage and egress. Pays off vs hosted API only at very high throughput or when data residency / open-weights are hard requirements. The economic case for K2.6 is strongest in two scenarios: high-volume coding agents where Opus is too expensive and DeepSeek's specialty does not match the workload, and regulated or air-gapped environments where you need open weights you can audit and self-host. For most teams, start on the hosted API and port to self-hosted only if economics or compliance demand it. ## How do you run Kimi K2.6 locally? K2.6 weights are on Hugging Face at [`moonshotai/Kimi-K2.6`](https://huggingface.co/moonshotai/Kimi-K2.6). The native INT4 quant is published directly by Moonshot — no community conversion needed. Community quants for llama.cpp, LM Studio, Jan, and Ollama landed within 48 hours of release. MLX builds for Apple Silicon followed about a week later. ### Hardware budget - **INT4, full 256K context:** 8x H200 141GB or equivalent ~640 GB aggregate VRAM. Verified target. - **INT4, reduced context (~32K):** 4x H100 80GB, or even a single H100 80GB for tight context windows. - **INT4 on 8x RTX 4090 (24GB each):** roughly 500 GB of VRAM with tensor parallel — works but tight. - **Storage:** INT4 weights are ~594 GB; budget at least 700 GB of fast SSD for the weights plus scratch. - **FP16 / BF16 full:** 1.5 TB system RAM plus one H100 via KTransformers CPU+GPU hybrid for offline / single-machine inference. ### vLLM path (production default) Pin `transformers>=4.57.1, <5.0.0` and use vLLM 0.19.1 (manually verified). Minimal command: ``` vllm serve moonshotai/Kimi-K2.6 \ --tensor-parallel-size 8 \ --max-model-len 262144 \ --enable-prefix-caching \ --quantization compressed-tensors ``` Prefix caching is required for any agent workload — without it you re-tokenise the system prompt every call and lose the price advantage. The official deploy guide on the Hugging Face model card has the full set of recommended flags for SGLang and KTransformers as well. ### llama.cpp path (local development) Community GGUFs are available within hours of any K2-class release; for K2.6 the `UD-Q2_K_XL` quant at ~350 GB is the recommended size/quality balance, and `UD-Q8_K_XL` is effectively lossless because Moonshot already uses INT4 for MoE weights and BF16 for everything else. Real but slow on a workstation; useful for development and offline tasks, not for serving production traffic. ### MLX path (Apple Silicon) The mlx-community has published `Kimi-K2.6-MoE-Smart-Quant` with per-component bit allocation tuned for the MoE + MLA architecture — effective ~4.5 bpw, near-6-bit quality at near-4-bit size. On a single Mac Studio M3 Ultra with 192 GB+, the model fits in roughly 150 GB. For full 256K context, two M3 Ultras over JACCL/RDMA provide the headroom. Throughput is ~5–15 tokens/sec depending on prompt length — fine for interactive development, not for production serving. ### SGLang and KTransformers SGLang is faster than vLLM on agentic workloads with shared-prefix patterns; the Claw Groups primitive specifically benefits from SGLang's RadixAttention. Same 8x H100 hardware target. KTransformers is the CPU+GPU hybrid path for offline / single-machine inference on a workstation with 1.5 TB RAM and one H100. For a wider walk-through of the open-weights operations side — model serving, evals, observability, autoscaling — see our [self-hosting LLMs complete guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/). ## What use cases is Kimi K2.6 best at? K2.6 was post-trained specifically for long-horizon agentic coding. The benchmark suite reflects that. The shape of work where K2.6 outperforms everything else is also the shape of work it was trained for. **Autonomous coding agents.** Multi-hour runs with many tool calls, branching plans, and recovery from failures. Moonshot's published reference run is a 12+ hour port of a Qwen 3.5-0.8B inference engine to Zig on a Mac, with 4,000+ tool calls across 14 iterations — throughput climbed from 15 to 193 tokens/sec end-to-end. This is the workload K2.6 is built for; no other open-weights model holds coherence past ~1,500 tool calls. This is the segment where K2.6's tool-invocation success rate (96.6%) and Claw Groups primitive both pay off. If you are building an agent that runs unsupervised overnight or across a weekend, K2.6 is the model to evaluate first. **Coding-driven UI/UX generation.** K2.6's 1,529 Elo on Code Arena WebDev sits #6 of 67 — ahead of every other open-weights model. The model ships front-end animations end-to-end from a prompt or a screenshot; the design mode specifically targets coding-driven UI flows rather than image-first design. **Long-context retrieval and synthesis.** 262,144-token window with MLA attention keeps KV memory tractable. Practical use: ingest an entire 200-file repo, ingest a multi-hundred-page RFC bundle, ingest a full year of meeting notes. Holds coherence at depths where K2.5 drifted. **Web research and multi-hop reasoning.** 92.5 F1 on DeepSearchQA and 86.3% on BrowseComp with swarms enabled are both leaderboard leaders. Combined with the 300-agent fan-out, K2.6 is the right model for "review every paper on this topic and produce a synthesis." **Multilingual and competitive coding.** SWE-bench Multilingual 76.7%, AIME 2026 96.4%, HMMT 2026 92.7%. Strong fits for international engineering teams and for any pipeline that includes competition-style algorithmic work. **Open-weights compliance environments.** Regulated industries that need weights they can audit, host on-prem, and freeze on a specific version. The Modified MIT license has effectively no commercial restriction below ~100M MAU. ## What are Kimi K2.6's limitations? Moonshot's launch post is, by frontier-lab standards, candid. The third-party picture matches: 1. **Output speed is mid-pack.** 34.4 tokens per second on the Moonshot API, ranked #46 of 83 models tracked by Artificial Analysis. Time-to-first-token is 3.04 seconds in thinking mode — meaningfully slower than Opus 4.7 (~1.2s) or GPT-5.5 (~0.8s). Instant mode roughly doubles throughput. 2. **Terminal-Bench gap is real.** 66.7% on Terminal-Bench 2.0 is a clear loss to GPT-5.5 (~82.7%). If your agents live in a shell rather than in code, K2.6 is not the right model. 3. **SWE-bench Verified still trails Opus 4.7 by ~7 points.** 80.2% vs 87.6%. On the hardest single-file bug fixes, Opus 4.7 still wins. K2.6 closes the gap on multi-file and multi-language work, but Verified is still Anthropic's home turf. 4. **Self-hosted INT4 has a transformers version pin.** `transformers>=4.57.1, <5.0.0`. Older inference stacks silently fall back to FP16 and OOM on single-node setups. This will catch a team off-guard the first time. 5. **Verbosity is high.** 170M output tokens during the Artificial Analysis Intelligence Index evaluation — among the chattiest models tested. Use instant mode or aggressive `max_tokens` caps for cost-sensitive workloads. 6. **Agent Swarms add overhead on sequential work.** Tasks that do not parallelise see 10–25% latency overhead from the swarm scheduler. Disable swarms (one `extra_body` flag) when you know the task is linear. 7. **Vision encoder is good, not great on stylised content.** MoonViT matches Opus 4.7 on dense documents and screenshots but lags on artistic / illustrative imagery. Not a typical agentic-coding concern, but worth knowing for design-tooling use cases. 8. **Hosted-API rate limits are conservative for now.** Default tier is 50 RPM; bumping to production tiers requires a request to `support@moonshot.ai`. Plan ahead if you intend to ship K2.6 to production traffic on day one. 9. **License has a scale clause.** Modified MIT, no restriction below ~100M MAU or $20M/month revenue; above that, you must display "Kimi K2" branding on the user interface. Most teams will never hit either threshold; for hyperscalers planning to embed K2.6 in user-facing products, it is a legal-review item before launch. 10. **Hosting region.** Moonshot's primary infrastructure is in Beijing; the .ai endpoint terminates in a US/EU edge but data eventually flows to Chinese-controlled infrastructure. For regulated workloads where data residency matters, self-host or use a Western re-seller (Cloudflare Workers AI, NVIDIA NIM, DeepInfra). ## How do you call the Kimi K2.6 API? You call K2.6 with the model id `kimi-k2.6` against the Moonshot platform API at `https://api.moonshot.ai/v1`. The API is OpenAI-compatible, which means existing OpenAI SDKs work out of the box. Key knobs: - **Thinking mode (default):** The model emits a hidden reasoning trace before the user-visible answer. Reasoning is returned in `response.choices[0].message.reasoning`; the visible answer is in `.content` as usual. Recommended temperature: 1.0, top-p: 0.95, `max_tokens`: up to 98,304 for hard reasoning. - **Instant mode:** Disable thinking with `extra_body={"thinking": {"type": "disabled"}}` for low-latency chat. Recommended temperature drops to 0.6. - **Preserve thinking across turns:** Set `thinking: {type: "enabled", keep: "all"}` to retain the full reasoning trace in multi-turn conversations. Useful for agentic loops; doubles the output-token bill. - **Multimodal:** Pass images as base64 PNG/JPG via `image_url`; pass MP4 video via `video_url` (official API only — vLLM and SGLang do not yet support video). - **Tool use:** Standard OpenAI tool-call schema. 96.6% tool-invocation success across the Moonshot tool benchmark. - **Agent Swarms:** Enable via `extra_body={"swarm": {"enabled": true, "max_agents": 300, "max_steps": 4000}}`. Set `enabled: false` on tasks you know are linear. A minimal Python example: ``` from openai import OpenAI client = OpenAI( api_key="sk-...", base_url="https://api.moonshot.ai/v1", ) response = client.chat.completions.create( model="kimi-k2.6", messages=[{"role": "user", "content": "Port this Python function to Rust."}], temperature=1.0, extra_body={"thinking": {"type": "enabled", "keep": "all"}}, ) print(response.choices[0].message.content) ``` ## Building with Kimi K2.6? Hire the engineers who know open-weights operations. K2.6 closes the gap between open weights and closed-frontier quality — but only if the team running it understands inference operations, agent design, and routing economics. Most teams that try to adopt open-weights models stall at the same three places: serving infrastructure (vLLM, SGLang, KTransformers, prefix caching), agent harnesses (MCP tools, evals, swarm scheduling), and the routing layer that sends the right task to the right model. [**Hire a Codersera-vetted AI or platform engineer**](https://codersera.com/hire?ref=kimi-k2-6-pillar) to stand up K2.6 in your stack: tensor-parallel deployments on H200 / H100, prefix caching, agent-swarm integration, eval pipelines, and a router that knows when to fall back to Opus 4.7 or GPT-5.5. Vetted, remote-ready, and available in days — not months. ## FAQ ### When was Kimi K2.6 released? Moonshot AI shipped Kimi K2.6 on April 20, 2026, nine months after the original K2 (August 2025) and five months after K2.5 / K2-Thinking (November 2025). ### What is the model id for Kimi K2.6 in the API? `kimi-k2.6`. Use it as the `model` parameter against `https://api.moonshot.ai/v1`. The API is OpenAI-compatible. ### What license is Kimi K2.6 released under? A Modified MIT license. Commercial use is permitted with no royalty. Above ~100M monthly active users or $20M/month revenue, you must display "Kimi K2" branding on the product UI; below those thresholds the license behaves as standard MIT. ### What hardware do I need to self-host Kimi K2.6? Full 256K context at INT4 needs 8x H200 141GB (or equivalent ~640 GB aggregate VRAM). Reduced context fits on 4x H100 80GB. Single H100 80GB serves INT4 at ~32K context. KTransformers CPU+GPU hybrid runs the FP16 weights with 1.5 TB RAM and one H100. Apple Silicon via MLX smart-quant fits in ~150 GB on a single Mac Studio M3 Ultra. ### How much does the Moonshot hosted API cost? $0.95 / $4.00 per million tokens (input / output) on Moonshot direct. Cache reads $0.16/M — an 83% input-side discount, applied automatically. OpenRouter is slightly cheaper at $0.74 / $3.50/M. Blended at a 3:1 input:output ratio: roughly $1.71 per million tokens. ### Does Kimi K2.6 support prompt caching? Yes, automatically. Repeated prefixes are cached and billed at $0.16/M on read. Unlike Anthropic's API, you do not need to mark cache-control points explicitly. ### Is Kimi K2.6 better than Claude Opus 4.7? For long-horizon autonomous coding, web research, multilingual coding, and competition math — yes. For SWE-bench Verified single-file bug fixes, graduate-level science, and dense MCP-tool workflows — Opus 4.7 still leads. On cost, K2.6 wins by 5–6x. ### Is Kimi K2.6 better than GPT-5.5? For SWE-bench Pro (tied at 58.6%), web research (DeepSearchQA 92.5 vs ~80), and competition math (AIME 2026 96.4% vs ~94%), yes. For the AAII composite (54 vs 60) and Terminal-Bench 2.0 (66.7% vs ~82.7%), GPT-5.5 still leads. K2.6 is roughly 3x cheaper. ### Is Kimi K2.6 better than DeepSeek V4 Pro? On long-horizon agentic work, agent swarms, multilingual SWE, and tool-use success rate — yes, decisively. On competitive programming (LiveCodeBench), V4 Pro leads (93.5 vs 89.6). SWE-bench Verified is a tie (80.6 vs 80.2). On raw cost, V4 Pro is about 4.6x cheaper on output ($0.87/M vs $4.00/M) at its standing price — narrowing to ~2x off-peak and roughly parity at peak once DeepSeek's August 16, 2026 price change lands. ### What are Agent Swarms and when should I use them? A native K2.6 primitive that lets the model spawn up to 300 sub-agents and coordinate up to 4,000 steps per task. Use them on parallelisable work — large literature reviews, multi-repo refactors, batch validation. Disable them on linear tasks where they add 10–25% latency overhead with no quality gain. ### Does Kimi K2.6 support vision and video? Yes. Images via base64 PNG/JPG (`image_url`) and MP4 video via base64 (`video_url`). Video input is currently official-API-only; vLLM and SGLang do not yet support it. ### Does Kimi K2.6 work with Cursor and Windsurf? Partly. As of early May 2026, Cursor has an open community request to add K2.6 to its model picker but no first-class integration — you can use it via OpenRouter or a custom OpenAI-compatible endpoint. Windsurf supports the K2 family in its picker; K2.6 specifically is rolling out. The wiring is identical to any other OpenAI-compatible endpoint either way. See our [Cursor IDE guide](https://codersera.com/blog/cursor-ide-complete-guide-2026/) for setup patterns. ### How do I disable the thinking trace for low-latency chat? Pass `extra_body={"thinking": {"type": "disabled"}}` on the official API, or `extra_body={"chat_template_kwargs": {"thinking": false}}` on vLLM / SGLang. Drop temperature to 0.6 in instant mode for best results. ### Where does Kimi K2.6 host its weights? Hugging Face at `moonshotai/Kimi-K2.6`. The native INT4 weights are ~594 GB. Community quants for llama.cpp, LM Studio, Jan, Ollama, and MLX are all available. =============================================================== [8/20] The Open-Source LLM Landscape in 2026 URL: https://codersera.com/blog/open-source-llms-landscape-2026/ Last updated: 2026-08-13 =============================================================== **Quick answer (refreshed June 16, 2026).** June 2026 brought a fresh open-weights wave on top of the May leaderboard. Headline shifts: **GLM-5.2** from Z.ai shipped with significant coding + agentic gains over 5.1 and was integrated into Nous Research's Hermes Agent within days — see our [GLM-5.2 complete guide](https://codersera.com/blog/glm-5-2-complete-guide-2026/). **Kimi K2.7 Code HighSpeed** claims 6× faster multimodal coding inference. **VibeThinker-3B** from WeiboAI — an MIT-licensed Qwen2.5-Coder-3B fine-tune — claims parity with frontier reasoners on math + code benchmarks at 3B parameters (see our [VibeThinker-3B guide](https://codersera.com/blog/vibethinker-3b-complete-guide-2026/) for honest framing). **Nemotron 3 Ultra** from Nvidia drops with what Sebastian Raschka calls an "ultra impressive capability:efficiency ratio". A new **Cohere 30B Command A+ derivative** arrived for agentic coding. On the hardware side, mid-2026 finally settled into a 4-way 128 GB-class showdown — DGX Spark $4k vs AMD Strix Halo vs RTX 6000 Pro $10k vs Macbook Pro M5 Max 128 GB $5k — see our [hardware showdown for June 2026](https://codersera.com/blog/local-llm-hardware-showdown-june-2026/). Per the LLM Stats Index, Claude Opus 4.8 retook #1 (closed) at 68; the open-weights gap to frontier is still inside 3 months. **August 13, 2026 update:** Meta reversed its closed-weights pivot — **Muse Glimmer** (30B dense, Apache 2.0, 128K context) shipped August 10 and runs on one 24–32 GB GPU or Mac, and Muse Spark 1.2’s weights are announced but not yet released. Details in the section below. ## August 2026 — Meta returns to open weights The biggest change to this landscape since June is that **Meta reversed its closed-weights pivot**. On August 10, 2026, Meta Superintelligence Labs released **Muse Glimmer**: a 30B dense decoder-only multimodal model (built-in vision encoder) under **Apache 2.0**, with a 128K context, text + image input, training across 100+ languages, and a lineage distilled from the closed Muse Spark line. Weights are ungated on Hugging Face under the `meta-models` org — roughly 121k downloads on the base repo and 352k on Unsloth's GGUF within days, with AMD, Arm, Dell, Hugging Face, Intel, llama.cpp, LM Studio, NVIDIA, Ollama, OpenRouter, Together AI, Unsloth and vLLM as launch partners. At ~4-bit the language model fits **under 20 GB**, so it lands squarely in the 24–32 GB single-GPU / Mac tier described further down (full precision needs >55 GB). A bundled DFlash speculative-decoding drafter measures 3.1x decode speedup on an RTX 5090, 1.8x on an M5 Max and 1.5x on an M4 Max. Hosted via OpenRouter as `meta/muse-glimmer-30b` at $0.35 / $1.50 per 1M. On quality, be honest. Meta’s own size-class table shows **Qwen3.6-27B beating Muse Glimmer on four of seven benchmarks** — SWE-Bench Verified 77.2 vs 76.0, TerminalBench 2.1 60.7 vs 51.7, OSWorld-Verified 75.6 vs 65.9, SkillsBench 46.6 vs 44.3. Glimmer’s clear wins are tool-calling shaped: MCP Atlas 75.5 vs 62.5, SWE-Bench Pro 51.2 vs 50.2, WildClawBench 47.6 vs 43.2. The defensible read — and the dominant one in the 1,199-point Hacker News launch thread — is that Glimmer is a **tool-use and agentic specialist in its size class** rather than a general coding leader. The other recurring critique is architectural: a *dense* 30B is memory-bandwidth-bound, so same-size MoE competitors often feel faster on the same hardware. Meta announced the same day that the weights for **Muse Spark 1.2** — the closed model behind its [Muse Code terminal agent](https://codersera.com/blog/muse-code-complete-guide-2026/), launched August 5 — will also be open-sourced, alongside a Zuckerberg essay arguing for American open-source AI (CNBC, FT and Fortune all covered it). Treat that as **announced, not shipped**: as of August 13, 2026 no Spark 1.2 weights exist on Hugging Face and no date has been given. For background on the closed half of the family, see our [Muse Spark complete guide](https://codersera.com/blog/muse-spark-complete-guide-2026/). ## June 2026 wave — what's new since May Three weeks after the May 27 refresh, the open-weights field had moved enough that we owe you a fresh layer. Below: the releases that materially changed the landscape between May 28 and June 16, 2026, with primary sources cited. - **GLM-5.2 from Z.ai (Zhipu)** — frontier intelligence with open weights, significant improvements over GLM-5.1 on coding and agentic tasks, strong long-horizon capabilities (per Z.ai's launch). Adoption velocity is the giveaway: Nous Research integrated GLM-5.2 into Hermes Agent within days, and OpenRouter routes it as `z-ai/glm-5.2`. Full deep dive with hands-on test results in our [GLM-5.2 complete guide](https://codersera.com/blog/glm-5-2-complete-guide-2026/). - **Kimi K2.7 Code HighSpeed from Moonshot AI** — a high-speed variant of the K2.7 multimodal coding model, claimed up to 6× faster inference at comparable quality. Builds on the K2.6 base that previously led HLE-with-tools. See the existing [Kimi K2.6 guide](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/); a K2.7-specific spoke is in the pipeline. - **VibeThinker-3B from WeiboAI** — an MIT-licensed, 3B-parameter fine-tune of Qwen2.5-Coder-3B specialised for math, code, and reasoning. WeiboAI's own claims (not the viral "3B = Opus" shorthand) are parity with DeepSeek V3.2 / GLM-5 / Kimi K2.5 on verifiable benchmarks like IMO-AnswerBench (76.4) and LeetCode (96.1% on Apr-May 2026 contests). 64K context, vLLM/SGLang runtime. The strongest local-first reasoning option in the sub-7B bracket. Full honest framing in our [VibeThinker-3B complete guide](https://codersera.com/blog/vibethinker-3b-complete-guide-2026/). - **Nemotron 3 Ultra from Nvidia** — open-weight release with what Sebastian Raschka ([@rasbt](https://x.com/rasbt), 666 likes) called an "ultra impressive capability:efficiency ratio". Continues Nvidia's pattern of open-weight Nemotron releases used as efficient student models distilled from larger teachers. - **Cohere 30B agentic coding model (Command A+ derivative)** — a new lightweight 30B open-weight specifically tuned for agentic coding tasks, builds on Command A+ using more recent training techniques. Strong low-VRAM agentic option per [@rasbt](https://x.com/rasbt) (744 likes). - **MiniMax M2 series technical report** — MiniMax shipped the technical report for the widely-used open-weight M2 series. The architecture deep-dives are useful even for teams not deploying M2 directly. [@rasbt](https://x.com/rasbt) covered the highlights (537 likes). - **Architecture: DeepSeek Sparse Attention (DSA) + Gated DeltaNet** — two attention-efficiency innovations that showed up in multiple June releases. [@rasbt](https://x.com/rasbt) shipped from-scratch PyTorch implementations of both in his LLMs-from-scratch repo. DSA cuts long-context KV-cache pressure further than the V3.2-era CSA/HCA tricks; Gated DeltaNet (used in Qwen3-Next and successors) is a "low-commitment" hybrid attention modification that can be slotted into existing transformer stacks during fine-tuning, not just pre-training. Both will likely show up in more flagships through 2026. ### June 2026 — the local LLM hardware picture finally settled Through April and May the local-LLM hardware story was confused by overlapping vendor announcements (AMD Strix Halo / Ryzen AI Max+ 395, NVIDIA DGX Spark, the M5 Max Macbook Pro). By June, practitioners with multiple boxes on their desks shipped enough honest benchmarks to make the four-way comparison usable. [@sudoingX](https://x.com/sudoingX) has both DGX Spark and Strix Halo on his desk (vendor-supplied for testing, disclosed); his finding that Vulkan beat ROCm by ~17% on token generation while tying on prompt processing is the kind of cross-vendor data that was missing through May. [@jun_song](https://x.com/jun_song) framed the three-tier matrix (RTX 6000 Pro $10k / DGX Spark $4k / M5 Max 128 GB $5k) and posted a sharp critique of the AMD AI Max+ 395 on actual bandwidth (~180 GB/s usable). Full synthesis with all 6 source tweets embedded in our [Local LLM Hardware Showdown — June 2026](https://codersera.com/blog/local-llm-hardware-showdown-june-2026/). The practical upshot for this pillar: hardware-tier recommendations below still apply, but the "single workstation" tier now genuinely has 4 credible 128 GB-class options at very different price points — pick by use case, not just budget. ## What's the May 27, 2026 top-5 open-weight leaderboard? | # | Model | Lab | Architecture | Index | Why it sits here | | --- | --- | --- | --- | --- | --- | | 1 | **Kimi K2.6** | Moonshot AI | 1T MoE, 32B active | 54 | Leads open Intelligence Index; ties GPT-5.5 on SWE-Bench Pro (58.6%) and leads HLE with tools (54.0%). | | 2 | **MiMo-V2.5-Pro** | Xiaomi | 1.02T MoE, 42B active | 54 | Surprise April 22 entrant. Leads Terminal-Bench 2.0 (68.4), 78.9 SWE-bench Verified, 1M context, 40-60% token-efficient vs Opus 4.6. | | 3 | **DeepSeek V4 Pro** | DeepSeek | 1.6T MoE, 49B active | 52 | Coding king. 80.6% SWE-bench Verified, 93.5% LiveCodeBench, MIT, $0.435/$0.87 — 35-86× cheaper than Opus 4.7. | | 4 | **GLM-5.2** | Z.ai (Zhipu) | ~750B MoE | ~52 | June 2026 successor to 5.1. Significant coding + agentic gains; integrated into Hermes Agent within days. MIT. See our [GLM-5.2 guide](https://codersera.com/blog/glm-5-2-complete-guide-2026/). | | 5 | **Qwen 3.6-35B-A3B** | Alibaba (open arm) | 35B MoE, 3B active | ~50 | Best efficiency story of 2026. GPQA Diamond 86.0, AIME 92.7, beats 397B dense rivals on coding while activating 3B per token. Apache 2.0. | Honourable mentions just outside the top-5: **DeepSeek V4 Flash** (284B MoE, ~$0.14/$0.28, cost-leader workhorse), **Mistral Medium 3.5** (128B dense, the only frontier-credible non-MoE, 77.6% SWE-bench Verified, 256K context), **Gemma 4 31B Dense** (Arena AI #3 open, leads small-footprint deployments), and **Llama 4 Maverick** (still the long-context leader for retrieval workloads at 1M tokens; Behemoth paused). ## What changed across the open-weights field in 2026? - **Muse Glimmer** (August 10) — Meta’s return to open weights. 30B dense multimodal, Apache 2.0, 128K context, ungated on Hugging Face, runs under 20 GB at 4-bit. Ends the “Meta has left open weights” read that held from April to August 2026. - **Mistral Medium 3.5** (April 30) — 128B dense, modified MIT, 256K context, replaces both Devstral 2 and Magistral in one model. Highest-quality non-MoE open model right now. - **GLM-5 (Feb 11) → GLM-5.1 (Apr 7)** — Z.ai's MoE family went from "interesting" to "topping SWE-Bench Pro" in 8 weeks. Trained entirely on Huawei Ascend hardware — proves non-NVIDIA frontier training is viable. - **Cohere Tiny Aya** (Feb 2026) — 3.35B params, 70+ languages, four regional variants (Global / Earth / Fire / Water). Research-only license, but the strongest multilingual story in the small-model bracket. - **Voxtral TTS** (March 26) — Mistral, 4B params, runs on 16 GB VRAM, zero-shot voice cloning from 3-5s audio. Beats ElevenLabs Flash v2.5 in 68.4% of blind A/B tests. CC-BY-NC. First credible open-weights TTS at production quality. - **MiMo-V2.5-Pro** (April 22) — Xiaomi's first frontier-credible release. Ties Kimi K2.6 on the Intelligence Index, ships fully open. - **No "Yi 2" yet.** 01.AI has not shipped a successor to the Yi family in 2026 — the slot they used to occupy is now held by Z.ai's GLM family and Xiaomi's MiMo. ## Is open-weights "winning" or quietly closing up? The honest read is mixed. **Alibaba flipped Qwen 3.7 Max closed** (API-only on DashScope, no weights). **Meta launched Muse Spark closed** in April 2026 as the first Meta Superintelligence Labs release — DeepLearning.ai called it an explicit pivot away from Llama's open strategy — and then **reversed that pivot on August 10, 2026**, shipping Muse Glimmer under Apache 2.0 and announcing that Muse Spark 1.2's weights will follow. **DeepSeek never wavered** — V4 Pro is MIT and the price discount went permanent. **Qwen still ships 3.6 open** under Apache 2.0 even while keeping 3.7 closed. **Z.ai, Xiaomi, Mistral, and Google (Gemma)** all shipped frontier-credible open weights in April-May. Epoch AI's data shows open-weight models now lag SOTA by ~3 months on average — the smallest gap ever measured. The honest summary: frontier labs treat open-weights as a strategic lever, not a principle. Labs without distribution moats keep shipping open; labs with their own distribution (Alibaba via Cloud) still hold back the very top tier. Meta is the proof that the lever swings both ways — it defected to closed weights in April 2026 and swung back in August — so the "open weights wins" narrative survives because new entrants and returning defectors keep replacing the labs that leave. ## When does self-hosting still beat the API in May 2026? Cost math has shifted hard. API providers commoditised inference faster than GPUs got cheaper. Rule of thumb: - **Under 1B tokens/day:** Use the hosted API of whichever open model fits the workload (DeepSeek for cost, Kimi for agents, Mistral for European compliance). - **1B–10B tokens/day:** Single-node self-hosting on 8× H100 or H200. Qwen 3.6-35B-A3B and Gemma 4 31B Dense are the most cost-effective at this band. - **Over 10B tokens/day, or sovereignty/compliance requirement:** Real cluster territory — DeepSeek V4 Pro on ~862 GB at INT4, or Kimi K2.6 on ~630 GB. Justified only at frontier-scale or where the API isn't allowed. For the deployment side — Kubernetes, vLLM, on-prem GPU planning — see our [self-hosting LLMs guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/). For Mac-native local inference of these models, see our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/). Per-model deep dives: [DeepSeek V4](https://codersera.com/blog/deepseek-v4-complete-guide-2026/), [Kimi K2.6](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/), [Qwen](https://codersera.com/blog/qwen-3-5-complete-guide-2026/), [Llama 4](https://codersera.com/blog/llama-4-complete-guide-2026/), [Gemma 4](https://codersera.com/blog/gemma-4-complete-guide-2026/). *Last updated: June 16, 2026.* The open-weight model landscape has changed more in the last 12 months than in the prior three years. DeepSeek pushed past 80% on SWE-bench Verified with weights anyone can download. Google shipped Gemma 4 under Apache 2.0. Alibaba's Qwen line went from "credible challenger" to "best-in-class on graduate-level reasoning." Meta's Llama 4 brought a 10M-token context window into the open. If you are still picking models based on a 2024 mental model, you are leaving capability, cost savings, and deployment flexibility on the table. This guide is the definitive landscape overview for engineering and product leaders evaluating open-weight LLMs in 2026. We cover the model families that matter, the architectures behind them, their license fine print, hardware tiers, and a decision framework you can actually use. Numbers throughout are from official model cards, the ArtificialAnalysis and LiveCodeBench leaderboards, and lab release blogs as of late April 2026. ## TL;DR - **The frontier is open.** DeepSeek V4-Pro hits 80.6% on SWE-bench Verified — within 0.2 points of Claude Opus 4.6 — under an MIT license. The capability gap between open and closed is now measured in single benchmark points, not generations. - **MoE is the default at scale.** Almost every flagship open model in 2026 is a sparse Mixture-of-Experts: DeepSeek V4-Pro (1.6T total / 49B active), Llama 4 Maverick (400B / 17B), Qwen 3.5 (397B / 17B), Mistral Large 3 (675B / 41B). Total parameters set the VRAM floor; active parameters set the inference cost. - **Apache 2.0 has won the license war for permissive labs.** Gemma 4, Qwen 3.5, Mistral Large 3, and Yi all ship under Apache 2.0. DeepSeek V4 ships MIT. Llama 4 keeps the Meta custom license with the 700M MAU clause. - **Context windows are no longer a gate.** Llama 4 Scout offers 10M tokens. DeepSeek V4 supports 1M. Gemma 4 medium offers 256K. The bottleneck is now retrieval quality and inference cost, not raw window size. - **Specialization beats raw size on benchmarks.** Qwen 3.5 leads open weights on GPQA Diamond at 88.4%. Gemma 4 31B hits 80% on LiveCodeBench at one-tenth the active params of frontier MoEs. Phi-4 14B beats 70B+ models on math reasoning. - **Hardware tiers are now well-defined.** 7B-14B runs on a 16GB GPU. 27B-32B needs an RTX 5090 (32GB) at Q4. 70B dense needs 48-64GB or dual cards. Frontier MoE (400B+) needs an 8-GPU H200/B300 box. - **The interesting question shifted** from "which model is best?" to "which model is best for this task at this latency budget under this license?" ## Why Open-Source Matters in 2026 The argument for open-weight models stopped being ideological a year ago. The drivers now are concrete: **Cost.** Hosted DeepSeek V4-Flash inference runs at a fraction of frontier closed-API pricing, and a self-hosted Gemma 4 4B can serve millions of internal requests for the price of a single GPU. For high-volume workloads — RAG over internal docs, code review bots, transcript summarization — the unit economics of open weights are not close. **Privacy and data residency.** Regulated workloads (health, finance, legal, defense) cannot send raw inputs to third-party APIs. Open weights run inside your VPC, on your hardware, against your data, with logs you control. This is the single biggest reason enterprise CTOs are funding GPU clusters in 2026. **Fine-tuning.** You cannot fine-tune Claude or GPT-5 on your proprietary corpus. You can fine-tune Qwen 3.5, Gemma 4, or Llama 4 in an afternoon with LoRA on a single H100. The closed labs offer fine-tuning APIs, but with strict data and policy constraints. **Vendor lock-in.** Anthropic, OpenAI, and Google have all changed pricing, deprecated models, and tightened policies on 30 days' notice in the last year. Open weights insulate your roadmap from another lab's product decisions. For a deeper look at the cost story specifically, see our [DeepSeek V4 Pro pricing review](https://codersera.com/blog/deepseek-v4-pro-review-benchmarks-pricing-2026/) and the broader [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). ## The Family Overview Matrix Here is the landscape in one table. "Active" means parameters used per forward pass on MoE models — what your inference cost actually scales with. "Total" sets the VRAM floor. | Family | Flagship (2026) | Total Params | Active | Architecture | Context | License | Released | | --- | --- | --- | --- | --- | --- | --- | --- | | DeepSeek V4 | V4-Pro | 1.6T | 49B | MoE | 1M | MIT | Apr 2026 | | DeepSeek V4 | V4-Flash | 284B | 13B | MoE | 1M | MIT | Apr 2026 | | Llama 4 (Meta) | Maverick | 400B | 17B | MoE (128 experts) | 1M | Llama 4 Community | Apr 2025 | | Llama 4 (Meta) | Scout | 109B | 17B | MoE (16 experts) | 10M | Llama 4 Community | Apr 2025 | | Muse (Meta) | Muse Glimmer 30B | 30B | 30B | Dense (multimodal) | 128K | Apache 2.0 | Aug 2026 | | Qwen 3.5 (Alibaba) | Qwen3.5-397B-A17B | 397B | 17B | MoE | 256K | Apache 2.0 | Feb 2026 | | Qwen 3.6 (Alibaba) | Qwen3.6-35B-A3B | 35B | 3B | MoE | 256K | Apache 2.0 | Apr 2026 | | Gemma 4 (Google) | Gemma 4 31B | 31B | 31B | Dense | 256K | Apache 2.0 | Apr 2026 | | Gemma 4 (Google) | Gemma 4 27B A4B | 26B | ~4B | MoE | 256K | Apache 2.0 | Apr 2026 | | Mistral Large 3 | Large 3 (2512) | 675B | 41B | MoE | 256K | Apache 2.0 | Dec 2025 | | Phi-4 (Microsoft) | Phi-4-reasoning-plus | 14B | 14B | Dense | 32K | MIT | 2025 | | Hunyuan (Tencent) | Hy3 Preview | 295B | 21B | MoE | 256K | Tencent Hy Community | Apr 2026 | | Yi (01.AI) | Yi 1.5 / Yi-Coder 9B | up to 34B | — | Dense | 200K+ | Apache 2.0 | 2024-2025 | | Falcon (TII) | Falcon H1R 7B | 7B | 7B | Dense (hybrid) | — | Falcon LLM 1.0 (Apache-derived) | Jan 2026 | | Cohere Command R+ | C4AI Command R+ | 104B | 104B | Dense | 128K | CC-BY-NC (research) | 2024 (still relevant) | | Stable LM 2 | Stable LM 2 12B | 12B | 12B | Dense | — | Stability Community | 2024 | ## The Benchmark Comparison Real numbers, not vibes. SWE-bench Verified is the truest signal for "can this model do real engineering work." LiveCodeBench is contamination-resistant competitive coding. GPQA Diamond is graduate-level science reasoning. MMLU-Pro is the harder, ten-option successor to MMLU. | Model | SWE-bench Verified | LiveCodeBench | GPQA Diamond | MMLU-Pro | | --- | --- | --- | --- | --- | | DeepSeek V4-Pro | 80.6% | ~93% (V4 Max variant) | ~80% | ~87% | | DeepSeek V4-Flash | 79.0% | ~88% | ~78% | ~85% | | Llama 4 Maverick | ~70% | — | — | 80.5% | | Llama 4 Scout | ~64% | — | — | ~74% | | Qwen 3.5 (397B-A17B) | 76.4% | 83.6% | **88.4%** | 87.8% | | Qwen 3.6-35B-A3B | 73.4% | ~80% | ~82% | ~84% | | Gemma 4 31B | ~68% | 80% | ~75% | 85.2% | | Mistral Large 3 | ~65% | ~70% | ~70% | ~78% | | Phi-4-reasoning-plus (14B) | — | — | 56.1% | ~74% | | Hunyuan Hy3 Preview | 74.4% | ~78% | ~75% | ~83% | | Falcon H1R 7B | — | — | — | — | A few observations from the table: - DeepSeek V4-Pro is the strongest open coder, closing in on Claude Opus 4.6. - Qwen 3.5 is the strongest open scientific reasoner, beating most closed models on GPQA Diamond. - Gemma 4 31B punches well above its weight class on coding — 80% LiveCodeBench from a 31B dense model is exceptional. - Llama 4 Maverick's MMLU-Pro of 80.5% beats GPT-4o, but it has fallen behind on coding benchmarks compared to the Chinese labs. - Phi-4 stays useful as a small reasoning specialist, not a generalist. For a focused head-to-head on the two most discussed open models of April 2026, see [Gemma 4 vs Llama 4 for local deployment](https://codersera.com/blog/gemma-4-vs-llama-4-local-deployment-2026/). For a deep dive on DeepSeek V4-Flash specifically, see the [V4-Flash deep dive](https://codersera.com/blog/deepseek-v4-flash-deep-dive/), and for a Gemma 4 standalone review including local-deployment specifics see [our Gemma 4 review](https://codersera.com/blog/google-gemma-4-review-benchmarks-features-run-locally/). One nuance worth flagging: benchmark scores are reported under different harnesses and prompt templates. A 2-3 point delta between the same model on two leaderboards is normal and usually reflects scaffold differences, not model quality. When you evaluate, fix the harness and the prompt and only then compare. The teams shipping the most impressive open-model deployments in 2026 are the ones who built rigorous internal evals before they touched production traffic. ## License and Commercial-Use Matrix This is where engineering teams get burned. "Open weights" is not the same as "open source," and "Apache 2.0" is not the same as "Llama 4 Community." Read the license before you ship. | Model | License | Commercial Use | Redistribution | Fine-tune & Derivatives | Notable Restrictions | | --- | --- | --- | --- | --- | --- | | DeepSeek V4 Pro / Flash | MIT | Yes, unrestricted | Yes | Yes | None of substance | | Qwen 3.5 / 3.6 | Apache 2.0 | Yes, unrestricted | Yes | Yes | Standard attribution | | Gemma 4 | Apache 2.0 | Yes, unrestricted | Yes | Yes | Standard attribution | | Mistral Large 3 | Apache 2.0 | Yes, unrestricted | Yes | Yes | Standard attribution | | Phi-4 | MIT | Yes, unrestricted | Yes | Yes | None of substance | | Yi 1.5 / Yi-Coder | Apache 2.0 | Yes, unrestricted | Yes | Yes | Standard attribution | | Llama 4 | Llama 4 Community License | Yes, with carve-outs | Yes | Yes | Companies with 700M+ MAU need a separate Meta license; products built on Llama must include "Built with Llama" attribution | | Muse Glimmer 30B | Apache 2.0 | Yes, unrestricted | Yes | Yes | Standard attribution; Meta's first fully permissive LLM release | | Hunyuan Hy3 | Tencent Hy Community License | Conditional | Conditional | Yes | Custom; review before deployment | | Falcon H1R | Falcon LLM 1.0 (Apache-derived) | Yes, royalty-free | Yes | Yes | No-litigate clause; attribution required | | Cohere Command R+ | CC-BY-NC 4.0 | **No** (research only) | Yes (non-commercial) | Yes (non-commercial) | Commercial use requires Cohere license | | Stable LM 2 12B | Stability Community | Requires Stability membership | Restricted | Yes | Membership tier gates commercial deployment | Bottom line for legal review: - If you want zero ambiguity: **DeepSeek V4 (MIT), Qwen 3.5 (Apache 2.0), Gemma 4 (Apache 2.0), Mistral Large 3 (Apache 2.0), Phi-4 (MIT)**. These are clean for any commercial deployment. - Llama 4 is fine unless you are at FAANG scale (700M+ MAU), but factor in the attribution requirement. - Cohere Command R+ is a research artifact. Do not ship it to production without a Cohere commercial agreement. - Tencent and Stability have custom licenses worth a 30-minute legal read before integration. ## How to Choose: A Decision Framework Walk down this list, in order. The first answer that matches your constraint picks the model. 1. **License must be permissive (legal/compliance gate).** Eliminate Cohere Command R+, Stability community-license models, and (if you are above 700M MAU) Llama 4. You are left with the DeepSeek/Qwen/Gemma/Mistral/Phi/Yi pool. 2. **Hardware budget.**Single 16GB consumer GPU: Phi-4 14B, Gemma 4 4B, Yi 9B, Falcon H1R 7B. 3. Single 32GB GPU (RTX 5090, A6000): Gemma 4 31B at Q4, Qwen 3.6-35B-A3B at Q4. 4. Dual 24GB or 48GB single card: Llama 3.3 70B class, dense 70B variants at Q4. 5. 8x H100/H200/B300 server: DeepSeek V4-Pro, Llama 4 Maverick, Mistral Large 3, Qwen 3.5-397B. 6. **Primary task.**Code generation / agent loops: DeepSeek V4-Pro > DeepSeek V4-Flash > Qwen 3.5 > Gemma 4 31B. 7. Scientific / graduate reasoning: Qwen 3.5 > DeepSeek V4-Pro > Gemma 4 31B. 8. Long-document RAG: Llama 4 Scout (10M ctx) > DeepSeek V4 (1M) > Gemma 4 (256K). 9. Tool-use / function calling: Qwen 3.5, Mistral Large 3, Llama 4 Maverick (all strong; pick on license). 10. On-device / edge inference: Phi-4 14B, Gemma 4 E2B/E4B, Falcon H1R 7B. 11. **Latency budget.** MoE models with smaller active-parameter counts (V4-Flash 13B active, Qwen3.6-35B-A3B 3B active) are dramatically faster than dense 70B models at similar quality. Active params, not total params, drive latency. 12. **Multilingual / regional needs.** Qwen and Hunyuan dominate Chinese. Mistral Large 3 is strongest on European languages. Llama 4 covers ~30 languages well. Yi is bilingual EN/ZH first. For a worked example of comparing two specific options, see [Gemma 3 vs Qwen 3](https://codersera.com/blog/gemma-3-vs-qwen-3-in-depth-comparison-of-two-leading-open-source-llms/) (much of the methodology carries forward to the 4/3.5 generation), and the [Hunyuan vs Qwen 3 comparison](https://codersera.com/blog/hunyuan-7b-vs-qwen-3-in-depth-comparison/). If you are already on DeepSeek and weighing alternatives, our [DeepSeek V4 alternatives](https://codersera.com/blog/deepseek-v4-alternatives-qwen-kimi-minimax-gpt-claude-compared/) piece walks through the trade-offs. ## Architecture Notes Worth Knowing **MoE is the dominant pattern at scale.** Every flagship open model above ~100B parameters in 2026 is a Mixture-of-Experts. The reason is straightforward: at training and serving scale, MoE lets you grow total capacity (knowledge, multilingual coverage, niche skills) without proportionally growing inference cost. DeepSeek V4-Pro with 1.6T total / 49B active runs faster than a hypothetical 70B dense model of similar quality. **Dense still wins under ~32B.** Below the MoE break-even, dense architectures are simpler to fine-tune, simpler to serve, and easier to quantize. Gemma 4 31B (dense) is the strongest sub-32B coder. Phi-4 14B (dense) is the strongest small-model reasoner. **Long-context tricks have matured.** DeepSeek V4 introduced Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), cutting KV cache to 10% of V3.2's at 1M tokens. Llama 4 Scout uses interleaved local/global attention for its 10M window. These are not free — recall quality at the tail of long contexts is still the main failure mode — but the engineering has stopped being the bottleneck. **Reasoning specialization is now a separate axis.** Phi-4-reasoning-plus, DeepSeek-R1-derived variants, and the Qwen3.6 reasoning lines are tuned with RL on chain-of-thought traces. They cost more tokens per answer but produce dramatically better math/science/code results. Treat reasoning models as a distinct product category from general chat models. **Hybrid attention and state-space ideas are creeping in.** Falcon H1R uses a hybrid Mamba/Transformer block, and several smaller labs are shipping pure SSM (state-space model) variants for very-long-context inference. None of these have yet displaced the Transformer-MoE flagship pattern at the top of the leaderboards, but they are showing up in the long tail and may matter more in 2027 if the trend continues. **Quantization is now part of the model release.** Most labs ship official Q4, Q5, and Q8 GGUF or AWQ quantizations alongside the FP16 weights. The days of waiting two weeks for the community to quantize a release are over — for production planning, assume you can deploy at Q4 the day a model drops. ## Hardware Tiers Cheat Sheet | Tier | VRAM | Example Hardware | What It Runs (Q4) | Approx. Cost | | --- | --- | --- | --- | --- | | Edge / laptop | 8-16 GB | RTX 4060 Ti 16GB, M3 Pro 18GB | Phi-4 14B, Gemma 4 4B, Yi 9B | ~$500-2,000 | | Power workstation | 24-32 GB | RTX 5090, RTX 4090 | Gemma 4 31B, Qwen3.6-35B-A3B, Muse Glimmer 30B | ~$2,000 | | Pro workstation | 48-96 GB | RTX PRO 6000 Blackwell, dual 5090 | 70B dense at Q4, 100B+ MoE at Q4 | ~$4,000-8,500 | | Single server | 1x 80GB H100/H200 | SXM5 H100/H200 | DeepSeek V4-Flash full-precision activations, Mistral Large 3 quantized | ~$25K-40K | | Frontier cluster | 8x H200 / B300 | NVL72, B300 server | DeepSeek V4-Pro, Llama 4 Maverick, Qwen 3.5-397B at production speed | $200K+ | Quantization is the lever that moves you down a tier. Q4_K_M cuts VRAM ~75% versus FP16 with minimal quality loss for most use cases. Below Q4 (Q3, Q2) quality starts dropping noticeably; only use it if you have nowhere else to go. ## What's Overhyped, What's Underrated **Overhyped:** - **10M context windows.** Llama 4 Scout's 10M token window is technically real, but practical recall past ~500K tokens is still shaky on every model we have tested. For real RAG workloads, retrieval quality + a 128K window beats raw 10M every time. - **Total parameter count as a marketing number.** "1.6 trillion parameters" sounds impressive; what matters is the 49B active and how cleanly the router picks experts. Compare on benchmarks, not headline counts. - **Reasoning-only specialist models for general workloads.** If you put Phi-4-reasoning-plus in a chat app, users will hate the latency. Use reasoning models where the user expects to wait — code agents, math/science assistants, planning loops. - **Falcon's "best 7B globally" claims.** Falcon H1R is good, but the AIME-only benchmark suite cherry-picks math reasoning. It does not generalize as cleanly to coding or open-ended tasks as the Phi-4 or Gemma 4 4B numbers suggest at similar size. **Underrated:** - **Gemma 4 31B.** A dense 31B model from Google under Apache 2.0 with 80% LiveCodeBench and 85.2% MMLU-Pro is the most practical "single GPU" deployment of 2026. Most teams should default to this and only move up the stack if they hit a real ceiling. - **DeepSeek V4-Flash.** Within 1.6 points of V4-Pro on SWE-bench at one-fifth the active params. For agent loops where you spend 30+ tool calls per task, the latency math overwhelmingly favors Flash. - **Qwen 3.6-35B-A3B.** 3B active parameters, 73.4% on SWE-bench Verified. The economics for self-hosted code review and PR triage are absurd. - **Yi-Coder 9B.** Quietly one of the best small coding models, 85% HumanEval at a size you can run on a Mac. - **Mistral Large 3 for European compliance.** French-headquartered, Apache 2.0, EU data residency story that DeepSeek and Qwen cannot match for some buyers. Often the only acceptable option for regulated EU customers. ## FAQ ### Is DeepSeek V4 actually open source, or just open weights? It is MIT-licensed, which is open source by the OSI definition for the released artifacts (weights, model code, inference code). The training data and training code are not released, so it is not "open" in the strictest reproducibility sense, but for downstream commercial use and modification, MIT is as permissive as it gets. ### Can I use Llama 4 in a startup? Yes. The Llama 4 Community License allows commercial use; the 700M MAU restriction only kicks in at hyperscale. You do need to add a "Built with Llama" attribution to your product and follow Meta's acceptable-use policy. Most startups can ship Llama 4 without legal friction. ### What is the cheapest model that is "good enough" for production code review? Qwen 3.6-35B-A3B or Gemma 4 31B. Both run on a single RTX 5090 at Q4, both clear 70%+ on SWE-bench Verified, both ship under Apache 2.0. Pick Qwen for stronger raw coding, Gemma for cleaner instruction-following. ### Does fine-tuning still matter, or is base capability good enough? Fine-tuning matters for domain language (legal, medical, internal jargon), tool schemas, and house style. It rarely improves raw reasoning. Use LoRA/QLoRA on a base open-weight model rather than full fine-tunes. ### What about multimodal models? Llama 4 is natively multimodal (text+image+video). Gemma 4 has vision-capable variants. Qwen ships dedicated VL models. For text-only workloads, ignore multimodal capability; it adds VRAM cost and complexity. ### Are Chinese-origin models safe for US enterprises? The model weights themselves carry no callback or telemetry — they are static files. Self-hosted Qwen, DeepSeek, or Hunyuan does not phone home. The risk surface is the API-hosted versions (where requests go to Chinese infrastructure). Your security/legal team should weigh in on training-data provenance and supply-chain considerations, but for self-hosted inference there is no technical leak path. ### How do I evaluate a new open model honestly? Run it on three things: (1) your real production prompts with held-out outputs, (2) a contamination-resistant benchmark like SWE-bench Verified or LiveCodeBench, and (3) a latency/cost test at your real concurrency. Public benchmarks are a filter, not a verdict. ### What is the difference between SWE-bench and SWE-bench Verified? SWE-bench Verified is a 500-task hand-curated subset of SWE-bench where each task has been confirmed to be solvable and to have correct test cases. It is the version everyone reports in 2026; raw SWE-bench numbers from 2024 are no longer comparable. ### Should I run inference myself or use a hosted open-weight provider? Hosted (Together, Fireworks, DeepInfra, Groq) wins for spiky workloads where you cannot keep a GPU warm. Self-hosted wins past ~10M tokens/day of steady traffic, where amortizing GPU cost beats per-token pricing. Run the math on your actual usage; do not assume. ### What about open-source reasoning/agent frameworks? Models are only half the story. Pair an open model with frameworks like vLLM (serving), SGLang (structured outputs), and LangGraph or smolagents (orchestration). The model choice and the harness choice are separate decisions. ### How fast is the open-source field moving in 2026? A new flagship-tier open model has dropped roughly every 4-6 weeks for the past year. Build your evaluation pipeline so swapping models is a config change, not a port. The team that can re-benchmark and ship the new model in a day captures the gains; the team that can't pays the closed-API premium another quarter. ### What's the right way to think about open vs closed in 2026? Closed (Claude, GPT-5, Gemini 3) still leads at the absolute frontier by 1-3 benchmark points and ships richer tool ecosystems. Open is "good enough" for the vast majority of production work, dramatically cheaper at scale, and the only option for privacy-sensitive deployments. Most serious teams now run a hybrid: open weights for high-volume internal workloads, closed API for the highest-stakes external user-facing surface. ### Where can I track this landscape day-to-day? The ArtificialAnalysis leaderboard, the LiveCodeBench leaderboard, Hugging Face's Open LLM Leaderboard v2, and the LMSYS / Arena rankings are the four dashboards worth bookmarking. Treat any single one as a single signal, not a verdict. ## Next Steps The fastest way to operationalize this landscape is to put it in the hands of someone who has shipped open-weight models in production — fine-tuned them, served them at scale, and made the latency/cost trade-offs in real systems. Models change every six weeks; the engineer who knows how to evaluate, deploy, and re-deploy them is the durable asset. [Hire a Codersera-vetted ML engineer](https://codersera.com/blog/hire/hire-python-developer) who has deployed open-weight models in production. Our developers have shipped DeepSeek, Qwen, Llama, and Gemma into live workloads — RAG systems, agent loops, on-prem inference clusters, fine-tuning pipelines — for teams that needed the speed and the privacy that closed APIs cannot give them. Skip the months of evaluation cycles and bring in the operator who has already done it. =============================================================== [9/20] Gemini 3.5 Complete Guide: Flash, Pro, Pricing & How It Compares (2026) URL: https://codersera.com/blog/gemini-3-5-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer.** Gemini 3.5 is Google DeepMind's mid-2026 frontier model family — **Gemini 3.5 Flash** launched at Google I/O on May 20, 2026 and is now the default Gemini in the consumer app, AI Mode in Search, and Vertex / Gemini API. **Gemini 3.5 Pro** was announced at the same event with full availability expected in June 2026. Compared to Gemini 3 Flash, the 3.5 generation brings stronger multimodal reasoning, sharper coding, lower API pricing (around $1.50 input / $9 output per 1M tokens for Flash), and tighter integration with the agent stack — particularly Google's renamed "Gemini Enterprise Agent Platform" (formerly Vertex AI Agent Builder). For most developers it competes head-on with Claude Sonnet 4.6, GPT-5.5, and DeepSeek V4-Flash; the Gemini differentiator remains native multimodality (text + image + audio + video in a single request), the largest production context window (up to 2M tokens on Pro), and tight Google Cloud + Workspace integration. This guide covers the lineup, benchmarks, pricing, ecosystem (AI Studio, Vertex, Gemini API, Gemini in Workspace, Gemini CLI), how to choose between Gemini 3.5 Flash and Pro, multimodal patterns that actually work, and how Gemini 3.5 compares to Claude Opus 4.7 and GPT-5.5 in practice. ## What is Gemini 3.5 and when did it launch? Gemini 3.5 is the May 2026 evolution of Google DeepMind's Gemini family. Two variants are shipping or imminent: - **Gemini 3.5 Flash** — launched at Google I/O 2026 on **May 20, 2026**. Replaced Gemini 3 Flash as the default in the Gemini app, in Google Search's AI Mode globally, and in the Gemini API (model id `gemini-3.5-flash`). Pricing announced: $1.50 per 1M input tokens, $9 per 1M output tokens — meaningfully cheaper than Gemini 3 at the same speed tier. - **Gemini 3.5 Pro** — announced at I/O, full general availability expected June 2026. Targets the frontier coding / reasoning / long-context use cases that previously needed Gemini Ultra. Context window of up to 2M tokens. The previous generation — Gemini 3 Flash — hit Q1 2026 with strong scores (GPQA Diamond 90.4%, Humanity's Last Exam 33.7%) and rapidly became the consumer default. Gemini 3.5 is the price/quality re-pivot on top of that base, with explicit positioning against Claude Sonnet 4.6 and GPT-5.5. Gemini 2.x and earlier are now legacy. The migration story is: Gemini 1.5 / 1.5 Pro → Gemini 2.0 (Feb 2025) → Gemini 2.5 Pro (mid-2025) → Gemini 3 (early 2026) → Gemini 3.5 (May 2026). Each step kept the API surface stable; model id changes plus pricing updates are the main migration work. ## How does Gemini 3.5 compare to Claude Opus 4.7 and GPT-5.5? The honest 2026 picture: three frontier families that trade places on different axes, with different strengths. | Dimension | Gemini 3.5 Pro | Claude Opus 4.7 | GPT-5.5 | | --- | --- | --- | --- | | Coding (SWE-bench) | Strong | **~80.8% SWE-bench Verified** (highest, Apr 2026) | Strong | | Long-context | **Up to 2M tokens** | 1M tokens (extended) | ~400K tokens | | Multimodal (text+image+audio+video) | **Native, single-request** | Text + image (strong) | Text + image | | Tool use / agents | Strong, Vertex / Agent Platform integration | **Computer-use agent, Agent Skills, Managed Agents** | Strong tool use, function calling | | Reasoning (extended thinking) | Yes | Yes (extended thinking) | **o3/o4-mini reasoning line is the leader** | | Pricing tier (frontier) | Mid (~$1.50/$9 Flash; Pro TBC) | Higher | Higher | | Free-tier access | **Generous** (AI Studio) | Limited (Claude.ai) | Free GPT-5.5 Instant | | Workspace / Office integration | **Native Google Workspace** | Via Claude for Work | Microsoft 365 Copilot | Where each one wins in 2026: - **Choose Gemini 3.5 when** you need the longest context (2M tokens), native multimodal (video + audio in one request), generous free-tier prototyping in AI Studio, or deep Google Cloud / Workspace integration. - **Choose Claude Opus 4.7 when** you need the strongest coding agent, computer-use capability, or the cleanest agent-development experience (Claude Code, Agent Skills, Managed Agents). See our [Claude Opus 4.7 guide](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/). - **Choose GPT-5.5 when** you need the o3/o4-mini reasoning ladder, Microsoft 365 integration, or you're already deep in the OpenAI ecosystem. See our [GPT-5.5 guide](https://codersera.com/blog/gpt-5-5-complete-guide-2026/). For agent workflows, Claude leads. For multimodal and ultra-long context, Gemini leads. For pure reasoning on hard problems, GPT-5.5's o3/o4-mini line is still the reference. Most production teams in 2026 use two or three of these via a model router rather than betting on one. ## What's new in Gemini 3.5 vs Gemini 3? The headline improvements Google demonstrated at I/O 2026: - **Cheaper Flash.** ~$1.50/$9 per 1M tokens — meaningfully below Gemini 3 Flash's prior tier and competitive with Claude Haiku 4.5 and GPT-5.5 Instant. - **Sharper coding.** Google demoed live multi-file refactors in the Gemini Code Assist extension and in the new Gemini CLI. Internal benchmarks show ~10-15 point gains on SWE-bench Verified vs 3.0 Flash. - **Better extended thinking** — Pro's reasoning chain is now visible to developers (similar to Claude's extended thinking and o3's reasoning traces) and can be capped to budget compute. - **Longer context with cleaner long-context recall.** 2M tokens on Pro with stronger needle-in-haystack performance than Gemini 1.5 Pro's earlier 2M context. - **Native multimodal in a single request.** Send an image, an audio file, and a video clip with one prompt — no separate API calls. The vision-audio-video joint reasoning is genuinely differentiated. - **Agent-platform integration.** Tighter coupling with Gemini Enterprise Agent Platform (the renamed Vertex AI Agent Builder), with first-class tool definitions, retrieval, evaluation, and deployment pipelines. ## How do I access Gemini 3.5? Five surfaces, depending on what you're building: **1. Gemini app (consumer).** The free Gemini at `gemini.google.com` and the mobile apps now run Gemini 3.5 Flash as the default. Useful for quick prototyping. **2. Google AI Studio.** Free developer playground at `aistudio.google.com`. Generous free tier — you can prototype real agents without a credit card. The best place to evaluate Gemini before committing to API. **3. Gemini API (direct).** SDK in Python, Node, Go, with REST as the universal fallback. Model ids: `gemini-3.5-flash`, `gemini-3.5-pro` (when GA). The simplest production path — no Vertex needed. **4. Vertex AI / Gemini Enterprise Agent Platform.** Enterprise hosting with IAM, VPC-SC, audit logs, fine-tuning workflows for the older Gemini variants, and the full Agent Builder pipeline. Use Vertex when you need GCP-native security and governance. **5. Gemini CLI.** Google's open-source command-line agent (similar to Claude Code, Cursor's CLI). One binary, MCP server support, runs against your API key. The 2026 entry point for terminal-first developers — see [github.com/google-gemini/gemini-cli](https://github.com/google-gemini/gemini-cli). For Workspace users: Gemini 3.5 is being rolled into Google Docs, Sheets, Gmail, Meet, and the Drive search experience. The "@Gemini" experience inside docs handles drafting, refactoring, multilingual translation, and the new "audio overview" feature for documents. ## What's Gemini 3.5 actually good at? **Long-context document analysis.** Drop a 1.5M-token document (a codebase, a legal contract, a 500-page PDF) and ask questions across it. No other major frontier model has this context length at usable recall. The 2026 sweet-spot use case: due diligence, code-base understanding, multi-document research synthesis. **Multimodal reasoning.** Send a screenshot of a UI + a screen recording + a spec doc and ask the model to write the fix. Gemini handles all three in one request, with cross-modal attention. The video-input quality genuinely beats GPT-5.5 in our testing — Google's video training data advantage shows. **Workspace-embedded productivity.** "Summarise this thread", "draft a reply", "build a sheet from this PDF" — the native Workspace surface is dramatically faster than the equivalent API workflow. For knowledge workers, this is the most-used Gemini surface. **Code review and refactor at file-tree scale.** With the 2M context, Gemini 3.5 Pro can hold a small codebase in mind and reason across it. Code Assist's "review this PR" with full repo context is meaningfully better than Gemini 3. **Real-time multilingual translation.** Gemini consistently leads on lower-resource language pairs, particularly Indian languages, Thai, Vietnamese, Indonesian, Swahili. If you're shipping product into India, Southeast Asia, or sub-Saharan Africa, Gemini's multilingual tail is the strongest of the three frontier families. Where Gemini 3.5 lags in 2026: agentic coding (Claude leads), terminal-first dev experience (Claude Code is more polished), the hardest reasoning benchmarks (GPT-5.5 with o3/o4-mini still wins on competition-math and the highest tiers). ## How do I structure my first Gemini 3.5 integration? Three patterns that work in 2026: **Pattern A — Direct API with the Python SDK.** ``` pip install google-genai from google import genai client = genai.Client(api_key=YOUR_API_KEY) response = client.models.generate_content( model="gemini-3.5-flash", contents=["Summarise this:", open("doc.pdf", "rb").read()], config={"thinking_config": {"thinking_budget": 8192}} ) print(response.text) ``` Use for: pure backend integration, no GCP dependency. **Pattern B — Vertex AI with IAM.** Same SDK, but authenticated via Application Default Credentials and routed through your Google Cloud project. Use when you need audit logs, VPC-SC, or you're already in GCP. **Pattern C — Gemini CLI as an agent shell.** Install with `npm install -g @google/gemini-cli` (or via brew). Authenticate with your API key. Run in a terminal — the CLI handles tool registration, MCP server discovery, multi-turn agent loops. Use for terminal-first development, code review, ops automation. See our [AI coding agents guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) for how it compares to Claude Code and Cursor. ## What does Gemini 3.5 cost? Gemini 3.5 Flash pricing announced at I/O 2026: **$1.50 per 1M input tokens / $9 per 1M output tokens**. Image and audio inputs priced as their token-equivalent. Free tier on AI Studio remains generous (60 requests per minute, with daily token quotas). Gemini 3.5 Pro pricing is to be confirmed at GA in June 2026; expected to follow the same ratio (~10× Flash) as prior Pro tiers, putting it in the $15/$60 per 1M tokens range — competitive with Claude Sonnet 4.6 and below Claude Opus 4.7 / GPT-5.5 Pro. Cached inputs: Google's prompt caching (released alongside Gemini 1.5 and improved with each generation) is the cost-control hammer for agent workloads. Cached tokens are charged at ~25% of input pricing. For any production agent loop that re-sends the same system prompt + tool definitions, prompt caching is the single highest-leverage optimisation. ## What about Gemma 4 (the open-weight side)? Gemini 3.5 is closed-weight, API-only. Google's open-weight family is the separate **Gemma** line — Gemma 4 launched April 2, 2026 with four sizes (E2B, E4B, 26B MoE, 31B dense) under Apache 2.0, 256K context, multimodal. Gemma is the right answer when you need to self-host, fine-tune locally, or run on edge devices. See our [Gemma 4 complete guide](https://codersera.com/blog/gemma-4-complete-guide-2026/). The pattern in 2026 for teams using both: Gemma 4 for self-hosted or fine-tuned workloads, Gemini 3.5 for frontier API tasks. They share lineage but serve different deployment models. ## FAQ ### Is Gemini 3.5 Pro actually released yet? Gemini 3.5 Flash launched at Google I/O 2026 on May 20, 2026 and is generally available now. Gemini 3.5 Pro was announced at the same event but full general availability is expected in June 2026. As of late May 2026, Pro is in limited preview for select Vertex customers. ### What's the context window? Gemini 3.5 Flash supports up to 1M tokens. Gemini 3.5 Pro supports up to 2M tokens — the largest of any production frontier model in May 2026. Long-context recall (the "needle in a haystack" benchmark) is meaningfully improved over Gemini 1.5 Pro at the same 2M cap. ### Is Gemini 3.5 free? Yes via Google AI Studio with a generous free tier (60 requests/min). The Gemini consumer app and AI Mode in Google Search use Gemini 3.5 Flash for free by default. Paid tiers exist for API usage and for the Gemini Advanced consumer subscription (Pro features in the app). ### Does Gemini 3.5 support function calling and tool use? Yes — function calling and structured outputs (with JSON schema validation) are first-class. The Gemini API exposes a `tools` parameter; Vertex / Agent Platform adds higher-level tool orchestration. For agent workflows, the integration with the renamed Gemini Enterprise Agent Platform (formerly Vertex AI Agent Builder) is the main 2026 surface. ### Can I fine-tune Gemini 3.5? Not the 3.5 generation directly as of May 2026 — Google offers fine-tuning on older Gemini variants (and Gemma family) through Vertex AI. For a fine-tunable Google-lineage model, use Gemma 4. See our [fine-tuning guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) for the broader landscape. ### How does Gemini 3.5 compare to Claude Opus 4.7 for coding? Claude Opus 4.7 leads on SWE-bench Verified (~80.8% in April 2026 — the highest in the category) and on agentic coding workflows via Claude Code. Gemini 3.5 Pro is competitive but lags by ~5–10 points on hard agentic tasks. For pure long-context code understanding (huge monorepo, multi-file analysis), Gemini's 2M context window gives it a different angle that Claude can't match in one request. ### What about Gemini's multimodal advantage? Genuine. Gemini accepts text, image, audio, and video in a single request with cross-modal attention. Video input quality in particular leads the field — Google's training data advantage shows. For workflows involving screen recordings, lecture videos, image+audio analysis, or any cross-modal reasoning, Gemini is the strongest choice. ### What's the cheapest way to evaluate Gemini 3.5? Google AI Studio (`aistudio.google.com`) — free, no credit card, full access to `gemini-3.5-flash`. Sufficient for production-scale prototyping. Move to the Gemini API once you exceed free quotas; move to Vertex when you need enterprise governance. ### Is there a Gemini equivalent to Claude Code or Cursor? Yes — the open-source Gemini CLI (`github.com/google-gemini/gemini-cli`), plus Gemini Code Assist as an IDE extension. The CLI is the closest analog to Claude Code; Code Assist competes with Cursor for IDE-integrated workflows. See our [AI coding agents guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) for the full comparison. ### What's coming next? Gemini 3.5 Pro full GA in June 2026. Expected: a Gemini 4 generation in late 2026 or early 2027 (no official roadmap), more specialised models in the Gemini family (a reasoning-focused variant similar to OpenAI's o3 line is widely rumoured), and continued Gemma 4 point releases on the open-weight side. ## Related guides - [Claude Opus 4.7 guide](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/) — the direct frontier competitor - [GPT-5.5 guide](https://codersera.com/blog/gpt-5-5-complete-guide-2026/) — the other major frontier - [Gemma 4 guide](https://codersera.com/blog/gemma-4-complete-guide-2026/) — the open-weight Gemini-family sibling - [AI coding agents](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) — Gemini CLI vs Claude Code vs Cursor - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) - [Fine-tuning LLMs](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) - [Apple Silicon LLMs](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) =============================================================== [10/20] AI Coding Agents in 2026: The Complete Guide URL: https://codersera.com/blog/ai-coding-agents-complete-guide-2026/ Last updated: 2026-08-13 =============================================================== **Quick answer (refreshed May 28, 2026).** May 2026 was the busiest month the agent category has ever had. **Cursor 3** (April 2) demoted the IDE to a fallback pane and shipped an agent-first interface; **Composer 2.5** (May 18) landed third on Artificial Analysis's Coding Agent Index at 62, behind only Claude Opus 4.7 in Claude Code (66) and GPT-5.5 in Codex (65), at roughly one-tenth the per-task cost. **OpenAI's Codex CLI** rewrote itself from TypeScript to Rust and shipped `/goal` persistent-thread Goal Mode to GA on May 21 (CLI 0.133). **Google announced Antigravity CLI** on May 19, replacing Gemini CLI; the old CLI stops serving free Pro/Ultra requests on June 18, 2026. **xAI's Grok Build** (May 14) shipped a terminal agent with up to 8 parallel sub-agents in isolated git worktrees, joined by **Grok Skills** (May 18) and **Connectors** (May 6 / May 22). **Claude Code** opened its plugin marketplace as a first-class system in spring 2026 (Opus 4.7 at 1M tokens), and the Anthropic web app got the sessions-sidebar redesign in April. **Cline 3.85** (May 25) added GPT-5.5 via SAP AI Core plus DeepSeek V4 Flash and Pro. And the standards layer: OpenAI and Anthropic co-founded the **Agentic AI Foundation** under the Linux Foundation, donating **AGENTS.md** (now in 60,000+ repos) and **MCP**. The picture below covers the May 2026 capability and pricing matrix; the rest of the guide covers per-tool deep dives. ## What's the May 2026 state of AI coding agents in one table? | Tool | Strongest at | Pricing (May 2026) | What changed since April | | --- | --- | --- | --- | | **Claude Code 2.1** | SWE-bench leader (80.8%), agentic loops, terminal-first | $200/mo Max; included with API spend | `/code-review`, Agent View, pinned bg sessions, plugin ecosystem, Opus 4.7 default | | **Cursor 3.5** | IDE-first, Cloud Agents, Composer 2.5 multi-file refactor | $20 Pro / $40 Ultra / $200 Max; Cloud Agents metered | 3.3 Build-in-Parallel + Jira; 3.5 Cloud Agents | | **GitHub Copilot agent mode** | VS Code integration, agentic code review, full-project context | $10–$39/mo; agent mode + BYOK now GA | BYOK third-party models, agent review | | **OpenCode** | OSS, Scout subagent, auto-compact, MCP-native | Free | 161K GH stars, Scout subagent | | **Cline** | OSS, BYO-API, MCP-native | Free (you pay API) | 61K GH stars, growing fast | | **Sourcegraph Amp** | Repo-graph semantic context, unconstrained tokens | Hosted SaaS, contact for pricing | Spun out as standalone company | | **Windsurf / Cascade** | IDE, Google AI behind the scenes post-acquisition | $15/mo Pro / $30 Ultimate | Founders → Google ($2.4B), rest → Cognition ($250M) | | **Gemini CLI** | Terminal-first, Gemini 3.5, free generous tier | Free with API key | Now defaults to Gemini 3.5 Flash; Pro coming June | | **Replit Agent** | Browser IDE, agent + deployment in one | Free + paid tiers | $400M Series D at $9B valuation (Mar 2026) | The right tool depends on workflow shape: terminal-first → Claude Code or Gemini CLI; IDE-first → Cursor or Cline; cloud-async → Cursor Cloud Agents or Replit Agent; OSS-only / BYO-key → OpenCode or Cline. For Cursor's specifics see our [Cursor IDE complete guide](https://codersera.com/blog/cursor-ide-complete-guide-2026/); for the underlying models, see [Claude Opus 4.7](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/), [GPT-5.5](https://codersera.com/blog/gpt-5-5-complete-guide-2026/), and [Gemini 3.5](https://codersera.com/blog/gemini-3-5-complete-guide-2026/). *Last updated: May 28, 2026.* The AI coding agent market doubled in size between mid-2025 and early 2026, and the field has finally fractured into recognisable categories: closed IDE-forks (Cursor, Windsurf), open IDE-forks (Void), terminal-native agents (Claude Code, Aider, OpenCode), VS Code extensions (Cline, Roo Code, Kilo Code, Continue.dev), and bring-your-own-key shells. Picking the wrong category costs hours of context-resetting before you even hit a paywall. This guide is the working comparison we use internally at Codersera when our vetted engineers onboard onto a new client codebase and need to recommend a tooling stack inside a week. We restrict the field to ten agents that have either real adoption (over 100k weekly active users) or a defensible architecture story: **Cursor, Claude Code, Cline, Aider, OpenCode, Continue.dev, Roo Code, Kilo Code, Windsurf, and Void AI**, plus three notable May 2026 entrants covered below: **OpenAI's Codex CLI** (Rust rewrite + `/goal` mode GA), **Google's Antigravity CLI** (Gemini CLI's successor), and **xAI's Grok Build**. We pulled pricing, model lists, and benchmark numbers from each vendor's docs, the SWE-bench Verified, SWE-bench Pro, Terminal-Bench 2.0, and Artificial Analysis Coding Agent Index leaderboards, and primary HN/Reddit threads from March through May 2026. Where vendors disagree with independent reports, we flag it. ## TL;DR - **Best default for a senior engineer in a real codebase:** Claude Code on Max ($100/mo) or Max-20x ($200/mo) with Claude Opus 4.7. It tops SWE-bench Verified at 87.6% (only Anthropic's still-private Claude Mythos Preview is higher at 93.9%) and SWE-bench Pro at 64.3%. The plugin marketplace, Skills, and 1M-token context on Opus 4.7 cemented its lead in spring 2026. - **Best closed IDE experience:** Cursor 3 ($20 Pro / $40 Ultra / $200 Max). Cursor 3 (April 2, 2026) moved the IDE behind an agent-first interface; **Composer 2.5** (May 18) is the new in-house model, ranked third on the Artificial Analysis Coding Agent Index at 62 and priced at $0.07–$0.44 per task — roughly one-tenth what Opus 4.7 or GPT-5.5 cost on the same harness. - **Best cloud-async pair:** **OpenAI Codex CLI with /goal Goal Mode** (GA May 21, 2026, in CLI 0.133). The Rust rewrite plus a persistent thread-level state machine means a single /goal directive survives network drops, closed laptops, and budget resets across hours-long sessions. - **Best new entrant for hardware-rich teams:** **xAI Grok Build** (early beta, launched May 14, 2026). Up to 8 parallel sub-agents each in its own git worktree, 256K context, 70.8% SWE-bench Verified. Access expanded May 24 to SuperGrok ($30/mo) and X Premium+ ($40/mo), down from the initial SuperGrok Heavy $300 gate. - **Best replacement for Gemini CLI:** **Google Antigravity CLI** (GA May 19, 2026). Built in Go, shares the agent harness with Antigravity 2.0 desktop. Free-tier Pro/Ultra access on the legacy Gemini CLI ends **June 18, 2026** — plan migrations now. - **Best open-source / BYOK:** Cline (now at 61k+ GitHub stars, GPT-5.5 and DeepSeek V4 Flash/Pro added in v3.85 on May 25), Kilo Code (1.5M users, 500+ models, zero markup), or Roo Code (forked from Cline, ~30% cheaper per task via diff-based editing). - **Best for privacy / fully local:** Continue.dev with Ollama (still actively shipping through May 2026), or OpenCode (v1.15.10 on May 27 — diff viewer in TUI, native API-key runtime, experimental background agents). - **Best terminal-only minimalist:** Aider. Pair it with Claude Opus 4.7, GPT-5.5, DeepSeek V4, or a local Qwen 3.5 model. Still no native MCP as of May 2026; community Aider-MCP server bridges exist. ## What changed in the agent landscape between mid-2025 and May 2026 Four things reshaped the field. First, **SWE-bench Verified saturated, but SWE-bench Pro is the number that matters.** On Verified, Claude Mythos Preview leads at 93.9%, followed by Claude Opus 4.8 (88.6%) and Claude Opus 4.7 Adaptive (87.6%); GPT-5.5 sits near 88.7%. But OpenAI's contamination audit showed every frontier model can reproduce verbatim gold patches on some Verified tasks because the 500 Python issues leaked into training data. On **contamination-resistant SWE-bench Pro**, Claude Mythos Preview leads at 77.8%, Opus 4.7 at 64.3%, and Qwen 3.7 Max at 60.6%; most agents drop 20+ points moving from Verified to Pro. Two newer benchmarks broke into the conversation in spring 2026: **Terminal-Bench 2.0** (89 hard end-to-end terminal tasks; GPT-5.5 leads at 0.827, Claude Mythos at 82.0%, GPT-5.3 Codex at 77.3%) and **Artificial Analysis's Coding Agent Index**, where Claude Opus 4.7 in Claude Code (max) sits at 66, GPT-5.5 in Codex (xhigh) at 65, and Cursor's Composer 2.5 at 62 — the cheapest agent above 60 by a 10–60× margin. Second, **MCP and AGENTS.md became governed standards.** In December 2025 the Linux Foundation formed the **Agentic AI Foundation (AAIF)** with OpenAI, Anthropic, and Block as founding members and Google, Microsoft, AWS, Bloomberg, and Cloudflare supporting. Anthropic donated the Model Context Protocol; OpenAI donated AGENTS.md, the project-level instruction file format now adopted by **60,000+ open-source repos and agent frameworks** (Codex, Cursor, Devin, Factory, Gemini CLI, GitHub Copilot, Jules, VS Code, Amp). For the AGENTS.md format itself, see our [AGENTS.md complete guide](https://codersera.com/blog/agents-md-complete-guide-2026/). By the close of Q2 2026, MCP had roughly **9,400 published servers across the four major registries and ~1,300 production-ready servers**, with the ecosystem mid-transition from stdio to hosted HTTP transport and from API keys to OAuth 2.1. Every agent in this guide except Aider now speaks MCP natively; community Aider-MCP bridges exist while official support remains on the roadmap. Third, **the agent surface moved out of the IDE.** Cursor 3 (April 2, 2026) reframed the editor as a fallback pane behind an agent-first interface, parallel agents across worktrees, and a Microsoft Teams integration. Claude Code's web app got a sessions-sidebar redesign in April with drag-and-drop layout and custom themes. OpenAI's `/goal` Goal Mode (CLI 0.133, May 21) is the same idea in CLI form: type a directive, the agent persists thread state through interruptions, network drops, and budget resets, and resumes when you're back. Google's Antigravity CLI (May 19) and xAI's Grok Build (May 14) both ship multi-agent orchestrators that run sub-agents in parallel against isolated git worktrees. The pattern is clear — the next twelve months belong to interfaces that let you supervise multiple agents rather than steer one. Fourth, **credit-based billing is collapsing back into flat or per-token pricing.** Cursor's credit pool remains the holdout but Composer 2.5's low per-task cost ($0.07 standard / $0.44 fast) makes the pool stretch much further; Windsurf moved to daily quotas with adaptive model routing; Cline, Kilo, and Roo all default to your own API keys at zero markup. The economics are converging on "you pay the model provider directly, the agent just orchestrates." For broader context on Cursor's UX tradeoffs versus the open camp, see our [Cursor vs Void comparison](https://codersera.com/blog/cursor-ai-vs-void-ai-an-in-depth-comparison-of-modern-ai-code-editors/) and the [Void privacy deep-dive](https://codersera.com/blog/void-ai-vs-cursor-features-privacy-local-models-limitations-2026/). ## What launched in May 2026: the month in detail The pace of change between May 1 and May 28, 2026 broke a record. Here's the timeline with the specific version numbers, dates, and sources so you can re-verify each line. ### Cursor 3 (April 2) and Composer 2.5 (May 18) Cursor 3 (April 2, 2026) shipped an agent-first interface that demotes the IDE to one of several panes. The new Agents Window runs many agents in parallel across repos and environments — locally, in worktrees, in the cloud, and on remote SSH. Cursor 3.3 (May 7) added a PR review experience inside Cursor with Reviews, Commits, and Changes tabs; Build-in-Parallel for plans; Dockerfile build secrets and 70%-faster layer caching; per-environment version history and audit logs; and a Microsoft Teams integration where you can `@Cursor` a cloud agent from any channel. **Composer 2.5** (May 18) is the new in-house coding model: it scored 62 on Artificial Analysis's Coding Agent Index (third place behind Claude Opus 4.7 in Claude Code at 66 and GPT-5.5 in Codex at 65), 79.8% on SWE-Bench Multilingual, and 63.2% on CursorBench v3.1 — and crucially priced at $0.50/M input and $2.50/M output (standard) or $3.00/$15.00 (Fast), about **10× cheaper** than the higher-effort variants ahead of it. Composer 2.5 also picked up **+35 points on SWE-Bench-Pro-Hard-AA**, +2 on Terminal-Bench v2, and +3 on SWE-Atlas-QnA versus Composer 2. ### OpenAI Codex CLI: Rust rewrite and /goal Goal Mode GA (May 21) OpenAI's Codex CLI completed its 2025–2026 rewrite from TypeScript to Rust; the Rust build now ships as the maintained CLI at versions 0.128–0.133 with new configuration, authentication, permission profiles, and sandbox modes. The headline ship of May 21, 2026 is **/goal Goal Mode GA** in CLI **0.133.0** and matching IDE/desktop builds. You type `/goal` followed by your prompt; Codex stores the goal as a persisted thread-level state machine and resumes work after network drops, deliberate pauses, or budget resets — including resuming a six-hour run after a five-hour pause. Earlier in the spring, OpenAI launched the **Codex plugin marketplace** (March 26, 2026, CLI 0.117) bundling Skills, MCP servers, and app connectors into shareable units; CLI 0.121 (April 15) added `codex marketplace add` for installing plugin marketplaces from GitHub, git URLs, or local directories. ### Google Antigravity CLI replaces Gemini CLI (June 18 cutoff) Google announced on May 19, 2026 that Gemini CLI is being sunset in favour of **Antigravity CLI**. Built in Go (snappier than the Node-based Gemini CLI), Antigravity CLI shares the same agent harness as Antigravity 2.0, Google's new desktop application — so future improvements land in both places. It orchestrates multiple agents for complex tasks in the background, runs large-scale refactors or research without locking the terminal, and supports agent skills, hooks, sub-agents, and extensions at launch. Antigravity CLI is not open source, which is a real change of posture from Gemini CLI. The **June 18, 2026 deadline** is when Gemini CLI and Gemini Code Assist IDE extensions stop serving requests for Google AI Pro, Ultra, and free users. Organisations on Gemini Code Assist Standard/Enterprise licences keep access; paid Gemini and Gemini Enterprise API key access is unaffected. If your team is on a free Pro plan, plan the migration this month. ### xAI Grok Build, Skills, and Connectors xAI ran the densest single month any vendor has shipped in this category. **Connectors** launched May 6, 2026, with first-wave integrations for GitHub, Notion, Linear, Google Workspace, Microsoft 365, plus Bring-Your-Own-MCP; May 22 added Vercel, Canva, Gamma, and S&P Global. **Grok Build** (May 14, 2026) is xAI's terminal coding agent with up to 8 parallel sub-agents, each in its own isolated git worktree across a three-stage plan/search/build workflow, 256K context, and a local-first privacy model that sends zero codebase data to xAI servers. It scored 70.8% on SWE-Bench Verified at launch. **Grok Skills** (May 18) is xAI's persistent-custom-expertise system: a Skill is a bundle of name, description, instructions, and optional reference files that you upload, then invoke via slash command or intent. API pricing is aggressive at **$0.20 input / $1.50 output per million tokens**, well under Opus 4.7's $5/$25. Access initially required SuperGrok Heavy at $300/mo; on May 24 it expanded to all SuperGrok ($30/mo) and X Premium+ ($40/mo) subscribers. For the deep dive see our [Grok Build, Skills, and Connectors guide](https://codersera.com/blog/xai-grok-build-skills-connectors-guide-2026/). ### Claude Code plugin marketplace and the April web redesign Anthropic's quiet ship of the spring was opening Claude Code's **plugin marketplace** as a first-class system. A Skill is a single instruction set; a plugin bundles multiple Skills, MCP servers, or commands. As of May 2026 both can be filtered in real time from `/skills` and `/plugin` prompts with type-to-filter. The official Anthropic marketplace sits at `claude-plugins-official`; the curated community list lives at `awesome-claude-code-plugins`. Claude Code on the web also got a redesign in Week 17 (April 20–24, 2026): a new sessions sidebar, drag-and-drop layout, custom themes, and a public research preview called `/ultrareview` with bug-hunting agents. Underneath all of this, **Claude Opus 4.7** moved native context from 200K to **1M tokens**, which is what makes the plugin/skill explosion practical inside a single Claude Code session. ### Cline 3.85, Windsurf, and OpenCode May updates **Cline 3.85.0** (May 25, 2026) added GPT-5.5 support via SAP AI Core, DeepSeek V4 Flash and Pro models, Gemini 3.5 Flash on the Gemini and Vertex providers, and routed Poolside Laguna models through next-gen prompts with native tool calling. The release also fixed Vertex AI's global endpoint handling for Claude models and exposed a manual update path that bypasses the release-age gate. **Windsurf** shipped Claude Opus 4.7 and GPT-5.5 in May, plus a Kanban-style view of local and cloud agent sessions, a Devin Local agent that's 30% more token-efficient than Cascade, Devin Cloud included with every self-serve plan ($50 of usage included on first cloud session), and an Adaptive model option in the picker that intelligently routes to keep quota alive. **OpenCode** shipped v1.15.6 on May 20 (diff viewer in TUI for reviewing changes, collapsed single-child directories, shell mode in the `run` prompt), v1.15.9 (redesigned diff viewer with file tree, clearer errors for invalid default models), and v1.15.10 on May 27 (restored legacy production desktop flows). Anthropic API-key models now use the native runtime, experimental background agents now push updates without polling, and the desktop added tabs, a native app menu on Windows, and Ukrainian locale support. ## Tool comparison matrix | Tool | Form factor | License | Agent loop | MCP | Best for | | --- | --- | --- | --- | --- | --- | | Cursor | VS Code fork (closed) | Proprietary | Composer / Agent | Yes | Frontend, fast feedback | | Claude Code | Terminal CLI + IDE plugins | Proprietary, source-available | Plan + Execute | Yes (remote MCP on Pro+) | Senior engineers, large refactors | | Cline | VS Code extension | Apache 2.0 | Plan / Act, human-in-the-loop | Yes | Auditable autonomy | | Aider | Terminal CLI | Apache 2.0 | Architect / Editor split | No (planned) | Git-native pair programming | | OpenCode | Terminal + desktop + ext. | MIT | Build / Plan modes | Yes | Privacy-first teams | | Continue.dev | VS Code + JetBrains ext. | Apache 2.0 | Chat + Agent | Yes | Enterprise, JetBrains shops | | Roo Code | VS Code extension | Apache 2.0 | Multi-mode (Architect, Code, Debug) | Yes | Cost-efficient agentic work | | Kilo Code | VS Code + JetBrains + CLI | Apache 2.0 | Subagents + Agent Manager | Yes | Heavy multi-agent workflows | | Windsurf | VS Code fork (closed) | Proprietary | Cascade | Yes | Codemaps + flow state | | Void AI | VS Code fork (open) | Apache 2.0 | Agent + Quick Edit | Yes | Local-only, privacy-strict | ## Pricing matrix (May 1, 2026) | Tool | Free tier | Individual paid | Team | Billing model | | --- | --- | --- | --- | --- | | Cursor | 2K completions | Pro $20, Pro+ $60, Ultra $200 | Business $40/seat | Credit pool = plan price | | Claude Code | None (Free Claude.ai excludes Code) | Pro $20, Max $100, Max-20x $200 | Premium $100/seat (annual) | Subscription + token caps; API per-token | | Cline | Extension free | BYOK (you pay model) | Cline Cloud (paid) | Pass-through | | Aider | CLI free | BYOK | n/a | Pass-through | | OpenCode | Free | Zen / Go credits optional | Self-host | BYOK or curated routing | | Continue.dev | Free | From $10/mo | Enterprise | Hub features + BYOK | | Roo Code | Free | Pro adds Roo Cloud | Team adds sync | BYOK | | Kilo Code | Free | Pay-as-you-go, zero markup | Same | Exact model price | | Windsurf | 5 daily AI interactions | Pro $15/mo | Teams $30/seat | Daily quota (post-March 2026) | | Void AI | Free | BYOK (or local) | n/a | Pass-through | Two effective-cost notes from real engineers running these in production: Cursor Pro's $20 credit pool buys roughly 225 Claude Sonnet requests, 500 GPT-4o requests, or 550 Gemini requests in agent mode. Roo Code's diff-based `apply_diff` tool reduces token spend by about 30% compared with Cline on equivalent tasks because it only emits changed lines in a 500-line file rather than the whole file. ## Claude Opus 5: the new near-frontier default (July 2026) Anthropic shipped **Claude Opus 5 on July 24, 2026** — its fourth model release in under two months and the one most likely to become the everyday default inside coding agents. Anthropic positions it as coming close to the frontier intelligence of the larger Claude Fable 5 *at half the price* ($5 input / $25 output per million tokens, the same as Opus 4.8 and half of Fable 5’s input cost). The headline for agent builders is agentic coding: on Frontier-Bench v0.1 (share of agentic coding tasks passed) Opus 5 scores **43.3%**, ahead of Fable 5 at 33.7% and GPT-5.6 Sol at 34.4% — an outright win, and more than double Opus 4.8 at a lower cost per task. It also tops the field on ARC-AGI-3 novel problem-solving (30.2% vs GPT-5.6 Sol’s 7.8%) and on the human-graded GDPval-AA v2 knowledge-work Elo (1,861), while landing under a point behind the SWE-bench Pro leaders at 79.2%. For agent loops specifically, two design choices matter. First, a per-request **effort toggle** (low / medium / high) lets you dial how much reasoning compute the model spends — low for routine tool calls to keep cost and latency down, high for the hard planning steps — which is exactly the cost/capability lever a multi-step agent wants. Second, Opus 5 leans hard on **self-verification and error recovery**: it checks its own work and iterates until a task succeeds without waiting for you to point out the mistake, so agents need less iterative back-and-forth per turn. It ships with a **1M-token context window** (that’s both the default and the maximum — there is no smaller variant), extended thinking on by default, and the API model id `claude-opus-5`, with the effort parameter and fast mode exposed on the platform. In practice you’ll meet it inside **Claude Code**, where a new **fast mode** ($10 input / $50 output, roughly 2.5x faster) trades price for speed on latency-sensitive work; it’s also the default on Claude Max and the strongest option on Claude Pro. Anthropic still steers the very longest-horizon autonomous runs (agents working for days) toward Fable 5, and frontier cybersecurity or the hardest biology work toward Mythos 5 — but for the coding-agent stack the tools below plug into, Opus 5 is the new sensible default. For the full spec sheet, benchmark tables, and pricing breakdown, see our [Claude Opus 5 launch guide](https://codersera.com/blog/claude-opus-5-launch-guide-2026/). ## Muse Code: Meta enters the terminal (August 2026) On **August 5, 2026**, Meta Superintelligence Labs shipped **Muse Code**, its first dedicated coding agent, alongside a coding-focused model update called **Muse Spark 1.2**. It is the first credible fourth entrant to the terminal-agent category that Claude Code, Codex CLI and Gemini's successor have shared for most of 2026. What it is: a single statically-linked binary (`muse`) installed with `curl -fsSL https://dev.meta.ai/install.sh | bash`. macOS and Linux only — native Windows is unsupported, WSL2 is the workaround. No Node.js or Python dependency. Version at launch was `0.1.0-R708.1`, explicitly a beta. **Where it is genuinely strong:** - **Safety defaults.** Approval prompts and an OS-enforced sandbox (Seatbelt on macOS, bubblewrap on Linux) are on from the first run, network defaults to `proxy-only`, and the sandbox *fails closed*. Inside the workspace, `.git`, `.muse` and `.agents` stay read-only. This is a stricter out-of-the-box posture than anything else in this guide. - **An append-only event log.** Every model call, tool run, approval and edit is journalled to `.muse/` before execution, giving replay-exact, restart-safe sessions and a genuine audit trail. Nothing else in the category ships this. - **Worktree-isolated subagents.** Children are checked out into their own git worktrees from the lead's commit, so parallel writes cannot collide. Concurrency is core count minus two, clamped 2–16; children run one level deep. - **Price.** $1.25 input / $4.25 output per million tokens on the standard tier — roughly 5.9x under Claude Opus 5 on output, with no strings. **Where it is behind:** no Windows build, no IDE extension, no subscription or spend cap, a single model with no fallback, and a day-old plugin ecosystem. It also loses on capability — on all three coding benchmarks *Meta itself published*, Claude Opus 5 finished first (Terminal-Bench 2.1: 86.7% vs 82.9%). Independent harnesses rank Muse Spark 1.2 lower still, at 14th of 50 on Vals AI's common scaffold. **The pricing story worth flagging to engineering leaders:** Muse Code also ships a *contributor tier* at $0.10/$0.20 per million tokens — 12.5x and 21.25x cheaper — in exchange for Meta training on your prompts and completions. It is selected by a model id rather than a signed agreement, and throttled to 60 requests per minute against standard's 3,000. Treat the tier choice as policy, not configuration. Two design decisions make it unusually easy to trial alongside an existing setup: it uses `AGENTS.md` as its project-rules file (falling back to `CLAUDE.md`), and `muse skills import --from claude` migrates existing skills in one command. It also scans repo-local `.claude/skills` and `.codex/skills`. **MCP is supported** over stdio and streamable HTTP — several launch articles reported otherwise, incorrectly. Deep dives: [the complete Muse Code guide](https://codersera.com/blog/muse-code-complete-guide-2026/), [install and setup](https://codersera.com/blog/how-to-install-muse-code-cli-2026/), [Muse Code vs Claude Code](https://codersera.com/blog/muse-code-vs-claude-code-2026/), [the benchmark reality check](https://codersera.com/blog/muse-spark-1-2-benchmarks-vs-claude-opus-5-2026/), and [what the contributor tier costs you](https://codersera.com/blog/muse-code-contributor-tier-privacy-2026/). ## Grok 4.6: xAI's August 2026 flagship xAI shipped **Grok 4.6** on **August 12, 2026**, 35 days after Grok 4.5 and a week after Muse Code. The API id is `grok-4.6`, the knowledge cutoff is February 1, 2026, and reasoning effort gains a third level — `xhigh` — above the low/high pair 4.5 exposed. It is already the default model inside **Grok Build** and shipped with a Cursor integration. Two naming clarifications, because both are circulating wrongly: there is no separate "Grok 4.6 High" model (that is the default reasoning effort), and the "2x price, faster" option is **Priority Processing**, a service tier, not a different SKU. xAI published no architecture details at all — no parameter count, no MoE-versus-dense answer, no compute figure, no system card. **Pricing is where agent builders should look hardest.** The sticker held at **$2 input / $6 output per million tokens**, but two things underneath it moved. Any prompt of **200K tokens or more bills the entire request at $4 / $12** — not just the overflow — a cliff a long-context agent loop can walk off without noticing. And cached input rose from $0.30 to **$0.50 per million (+67%)**, which bites precisely because agents re-send the same system prompt and file context every turn. Artificial Analysis measured the combined effect as a **2.32x rise in cost per task** — $0.360 on 4.5 to $0.837 on 4.6 — from roughly 47% more output tokens plus the cache increase. The per-token price held; the per-job price did not. **The genuine win is turn efficiency.** On Artificial Analysis's Briefcase agentic suite Grok 4.6 finishes in about **53 turns and 0.5B input tokens** where Claude Opus 5 takes ~103 turns and 2.0B — roughly 4x cheaper on the same work. That is what the early consensus has settled on: plan with an expensive model, let Grok implement. *Cheapest credible implementer* is the defensible framing, not *frontier*. On composite boards it sits 4th on the AA Intelligence Index at **60.92** (xAI claimed 61 — a clean vendor claim), behind Claude Opus 5, Claude Fable 5 and GPT-5.6 Sol Max, and 6th of 46 on the Vals Index at 71.82, a hair behind Muse Spark 1.2's 71.88. The two neutral harnesses disagree on that last ordering, which is itself worth knowing. **Where it regressed, and the benchmark trap.** LiveBench *agentic* coding fell to **54.2 from Grok 4.5's 56.5** even as non-agentic coding rose 6.4 points — so the regression is specific to the tool-using loop this guide is about. SkillsBench dropped from 66.03 to 55.77, time-to-first-token went from 8.7s to 31.2s, and 65.5 tok/s ranks 17th of 20 measured models; this is not a snappy interactive model. On Terminal-Bench, read the version label before the number: xAI reports **v3.0 at 26%**, while independent **v2.1** runs land at 88.39% (Artificial Analysis) and 78.28% (Vals). Those are different benchmarks and must never be compared, and no Grok 4.6 run has been submitted to the official verified board. Two hard constraints for teams: the **500K context window is the smallest of any current frontier model** (the rest are 1M+), and the API serves from **us-east-1 and us-west-2 only — there is no EU region**, which rules it out of some data-residency setups outright. Full spec sheet and benchmark tables: [our Grok 4.6 launch guide](https://codersera.com/blog/grok-4-6-launch-guide-2026/). ## Model support matrix | Tool | Frontier closed models | Open / local | BYOK | Notable defaults | | --- | --- | --- | --- | --- | | Cursor | Claude Opus 4.7, Sonnet 4.6, GPT-5.5, Gemini 2.5 Pro | Limited | Yes (custom) | Auto mode (router) | | Claude Code | Claude Opus 4.7, Sonnet 4.6, Haiku 4.5 | No | No (Anthropic-only) | Sonnet 4.6 default, Opus on Max | | Cline | Anthropic, OpenAI, Gemini, Bedrock, Vertex | Ollama, LM Studio, OpenAI-compatible | Yes | VS Code LM API (experimental) | | Aider | Claude 3.7+/4.x, GPT-4o/5, DeepSeek V4, o-series | Ollama, OpenAI-compatible | Yes | Architect/Editor pair | | OpenCode | 75+ providers | Ollama | Yes | Zen routing optional | | Continue.dev | OpenAI, Anthropic, Azure, Bedrock | Ollama, vLLM, TGI | Yes | Hub for shared configs | | Roo Code | OpenRouter (300+ models) | Ollama, LM Studio | Yes | Custom modes per model | | Kilo Code | 500+ models via OpenRouter and direct | Ollama, LM Studio | Yes | Subagents auto-delegate | | Windsurf | SWE-1.5 (in-house), Claude, GPT-5 | Limited | Partial | Cascade with Codemaps | | Void AI | Anthropic, OpenAI, Gemini | Ollama, LM Studio, DeepSeek, Qwen, Llama | Yes | Local-first defaults | For setup walk-throughs on the open / local side, see [Qwen 3.5 + Claude Code OSS](https://codersera.com/blog/run-install-and-benchmark-qwen35-claude-code-free-local-ai-coding-agent/), [OpenClaw + Ollama](https://codersera.com/blog/openclaw-ollama-setup-guide-run-local-ai-agents-2026/), and our deep dives on [DeepSeek V4](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) and the cheaper [DeepSeek V4 Flash](https://codersera.com/blog/deepseek-v4-flash-deep-dive/). ## Agent loop architectures, compared "Agent loop" is the structural difference that decides whether a tool can survive a multi-hour task. Three patterns dominate in 2026: **Plan / Act split (Cline, Roo, OpenCode).** The model first emits a plan with no file mutations. The user approves or edits the plan. Only then does the agent transition to Act mode where it can write files and run shell commands. Cline pioneered "human-in-the-loop" — every file edit, every command, every browser action requires explicit approval. That makes it slower but auditable, which matters when an agent is touching production code or migrating a database schema. Roo Code keeps the same skeleton but adds five built-in modes (Code, Architect, Ask, Debug, Custom) and uses diff-based edits to cut token cost. **Architect / Editor pair (Aider).** A reasoning model (o1, DeepSeek R1, Opus) drafts the change in plain English. A faster, cheaper editor model (Sonnet, GPT-4o, DeepSeek V3) translates the plan into precise diffs. The split costs more per turn but is the most reliable single pattern for large refactors because the planner never spends tokens on syntax. **Subagents and orchestrators (Claude Code, Kilo Code).** The top-level agent spawns specialised children — a "test runner" subagent, a "schema migration" subagent, a "frontend styling" subagent — each with its own context window. Kilo Code's April 2026 rebuild made this the headline feature: parallel tool calls and an Agent Manager that runs multiple agents side by side. Claude Code does the same via its Task tool. The downside is debuggability; when something goes sideways inside a subagent you have less visibility than a flat plan/act trace. For a hands-on look at running the open Claude Code internals, our [Claude Code OSS guide](https://codersera.com/blog/how-to-run-open-source-claude-code-oss-developer-guide-2026/) walks through the orchestrator, and [using Claude 4/Sonnet with Cursor and Windsurf](https://codersera.com/blog/how-to-use-claude-4-and-sonnet-with-cursor-and-windsurf/) covers the closed-IDE side. ## Real workflow examples ### Greenfield: a Next.js 15 app with Postgres and auth This is the case every demo nails. Cursor, Windsurf, Claude Code, and Cline all produce working scaffolds in under 15 minutes. The tiebreaker is what happens when you ask for a non-trivial second feature on top — say, "add Stripe Connect with webhook signature verification and idempotency keys." Claude Code on Sonnet 4.6 produced the cleanest output in our internal test (3 files modified, 1 webhook signature bug caught before commit). Cursor's Composer was 2x faster but missed the idempotency key on first pass. Aider with Architect + Editor produced the smallest diff but required a manual `/add` for the migration file because Aider's repo map didn't pull it in automatically. ### Large-codebase refactor: migrate 80 files from Redux to Zustand This is where most agents fall apart. The honest results from a 90k-line internal client codebase, March 2026: - **Claude Code with Opus 4.7 + subagents:** finished in 4 hours over 3 sessions, 76 files cleanly migrated, 4 needed manual fixes. Cost: ~$38 of API + Max subscription. - **Cline with Sonnet 4.6:** finished in 6 hours but the human-in-the-loop confirmations were the bottleneck. 78 files cleanly migrated. Cost: ~$22 BYOK. - **Cursor Agent (Auto):** blew past the credit pool at file 31, then degraded to GPT-4o-mini and produced inconsistent type imports. 51 files cleanly migrated, 29 needed rework. - **Aider with Opus 4.7 architect + Sonnet 4.6 editor:** 71 files cleanly migrated. The repo map saved time but Aider needed explicit `/add` calls for cross-package files. - **Roo Code with Sonnet 4.6:** 73 files migrated, ~30% cheaper than Cline thanks to diff-only edits. ### Debugging: tracking down a flaky test Aider, Cline, and Claude Code dominate this category because they can run the failing test in a loop and read the output. Aider's tight loop (run tests, read errors, re-edit) is still the fastest. Cursor's agent can do this too but the IDE chrome adds friction. Continue.dev with a local Qwen 3.5 model on Ollama handled a flaky pytest fixture without any cloud round-trip — slow (~45s per turn) but completely private. ## Privacy, deployment, and the local-models story Three of the ten agents are credible for "no code leaves my machine": Continue.dev with Ollama, Void AI with local DeepSeek/Qwen/Llama, and OpenCode (which explicitly states it stores no code or context). Cline and Kilo can be configured local-only but their default user flow assumes a cloud model. Aider is local-capable via Ollama but its Architect mode realistically requires a frontier model to be useful. If your bar is "compliance-grade local," Continue.dev plus a 70B-class local model is the production-tested combo. Void AI is technically excellent but the team announced an active-development pause in early 2026 — the binary still works and the repo is open, but you should not assume new features land in 2026. For teams deploying agents into regulated industries — EU AI Act, MiCA, NIST AI RMF — there is now a separate governance layer worth knowing about: [cryptographic identity for AI agents](https://kakunin.ai/), including X.509-backed Non-Human Identity, behavioral monitoring, and audit reports designed for the same compliance frameworks that gate frontier-model deployment. ## Known issues - **Cursor credit-pool surprises (mitigated by Composer 2.5).** Heavy agent-mode users on the $20 Pro plan still burn the pool fast, but Composer 2.5's $0.07–$0.44 per-task pricing makes the pool stretch much further than it did under the GPT-5.5 / Opus 4.7 era. If you need frontier-model agent work, Pro+ at $60 or Ultra at $200 is still the realistic tier. - **Gemini CLI free Pro/Ultra access ends June 18, 2026.** Migrate to Antigravity CLI before that date or lose access mid-task. Antigravity CLI is not open source, unlike Gemini CLI — a real change in posture if you care about auditability or self-hosting. - **Goal Mode is GA but session storage is OpenAI-side.** `/goal` persists the directive on OpenAI's infrastructure; if you have a no-data-leaves-our-VPC policy, Codex CLI's Goal Mode doesn't fit. Local-first alternatives are Aider's tight test loop or Claude Code on a self-hosted Bedrock/Vertex endpoint. - **Grok Build is early beta.** 8 parallel sub-agents is impressive on paper but the beta has rough edges around long-running tasks; xAI has acknowledged this and committed to daily release notes. The local-first privacy model is real, but the rapid release cadence means you should pin specific versions in CI. - **Claude Code on Pro is still conservatively rate-limited.** Long-running Opus 4.7 sessions on 1M-token context hit the Pro cap quickly. Max ($100) or Max-20x ($200) is the realistic floor for senior engineers using it as their primary agent. - **Cline's human-in-the-loop is slow on long tasks.** The Auto-approve toggle helps but defeats the audit story. Roo Code's batched approvals are a better compromise. - **Aider still has no native MCP.** A community Aider-MCP server bridges via WebSocket (allowing Claude and other MCP clients to call Aider tools), but Aider itself doesn't yet speak MCP to outside servers. If you depend on internal MCP tools, Aider is still effectively out. - **Windsurf has Devin Cloud now but pre-March-2026 credit-billed accounts** don't get Supercomplete or new SWE-1.5-tier features. New users start on daily-quota billing; the Adaptive model option helps keep quota alive. - **Void AI development is still paused.** The editor functions but there's no roadmap. Treat it as "stable open-source artifact," not "active product." - **SWE-bench Verified contamination persists.** Use SWE-bench Pro or Terminal-Bench 2.0 when evaluating models. The 87.6% Verified score for Opus 4.7 drops to 64.3% on Pro; the 93.9% Mythos Preview Verified drops to 77.8% on Pro. - **OpenCode and Kilo Code overlap with Cline genealogically.** All three share Apache-2.0 lineage; if you have a hard organisational requirement against forks-of-forks, audit dependencies before standardising. - **Antigravity 2.0 / Cursor 3 / Grok Build all bet on parallel sub-agents in worktrees.** Debuggability is the universal weak point — when something goes wrong inside a sub-agent, traceability is worse than a flat plan/act trace. Build observability into your AGENTS.md and sub-agent prompts before relying on this at scale. ## How to choose Four questions decide it for most teams in mid-2026: 1. **Does your codebase exceed 50k lines?** If yes, you need a planner/architect step (Aider Architect, Claude Code sub-agents, Roo Code's Architect mode, Grok Build's plan/search/build sub-agents) or a 1M-token context model like Opus 4.7. Pure inline-completion tools like Cursor's basic Tab degrade fast on large repos. 2. **Can your code legally leave the machine?** If no, your shortlist is Continue.dev + Ollama, OpenCode + local, Grok Build (local-first, zero codebase data sent to xAI), or Void AI + local. Everything else assumes cloud. 3. **Do you need sessions that survive interruptions?** If you run hours-long tasks on flaky networks or have to step away mid-run, Codex CLI `/goal` is the only GA tool with persistent thread-level state at the protocol level. Cursor 3 Cloud Agents and Antigravity CLI background runs are the IDE-side equivalents. 4. **Who owns the bill?** If the company pays per-seat predictably: Cursor Business, Windsurf Teams, Claude Code Premium, or Codex Enterprise. If individual engineers expense API: Cline, Aider, Roo, or Kilo with BYOK — and for cheap agentic work specifically, Cursor with Composer 2.5 at $0.07–$0.44/task. ## Frequently asked questions ### Which AI coding agent has the highest SWE-bench Pro score in 2026? As of May 2026, Claude Mythos Preview leads SWE-bench Pro at 77.8%, Claude Opus 4.7 at 64.3%, and Qwen 3.7 Max at 60.6%. GPT-5.4 (xHigh) reaches 59.1% on Scale's SEAL mini-swe-agent scaffold and GPT-5.3-Codex 56.8%. On the older SWE-bench Verified, Claude Mythos Preview hit 93.9%, Opus 4.8 88.6%, Opus 4.7 Adaptive 87.6%, and GPT-5.5 around 88.7% — but Verified is contaminated and Pro is the more honest comparison. ### Is Cursor still worth $20/month in 2026? More than before, because of Composer 2.5. Composer 2.5 (May 18, 2026) is the in-house model that ranks third on Artificial Analysis's Coding Agent Index at 62, and per-task pricing is $0.07 (standard) or $0.44 (Fast) — 10–60× cheaper than the higher-effort Opus 4.7 and GPT-5.5 variants above it. For daily agent-mode work on Composer 2.5 the $20 Pro pool now stretches a long way. If you specifically need Opus 4.7 or GPT-5.5 for the hardest tasks, Pro+ at $60 or Ultra at $200 is still the realistic tier. ### What's the difference between Claude Code and Cline? Claude Code is a terminal-native CLI from Anthropic, locked to Anthropic models, with subagents and tight Sonnet/Opus integration. Cline is an open-source VS Code extension that works with any model provider. Claude Code is more polished and faster on Anthropic infra; Cline is more flexible and free to install. ### Does Aider support MCP? Not natively as of May 2026. Native MCP support is on the roadmap but not shipped. The pragmatic workaround is the community Aider-MCP server (exposing Aider's edit_files, create_files, git_status, and similar tools to Claude and other MCP clients via WebSocket); this lets MCP clients drive Aider rather than Aider drive MCP servers. For most users the substitute is custom slash commands and the `/run` primitive. ### Are Roo Code and Cline really that similar? They share genealogy — Roo Code forked from Cline — but Roo added custom modes, diff-based editing, and broader model support. Independent measurements show ~30% cost savings on equivalent tasks because Roo's `apply_diff` only emits changed lines. ### What is Kilo Code and how does it differ from Roo and Cline? Kilo Code began as a fork of Cline and rebuilt itself in April 2026 onto a portable open-source core that ships across VS Code, JetBrains, CLI, mobile, and Slack. It's now a multi-agent platform with subagents and an Agent Manager. With 1.5M users and access to 500+ models at zero markup, it's the heaviest of the three. ### Can I use AI coding agents fully offline? Yes, with Continue.dev plus Ollama, OpenCode plus a local model, or Void AI plus Ollama/LM Studio. Realistically you'll want a 70B-class quantised model on a workstation with 64GB+ RAM or a Mac Studio. ### Which agents support MCP servers? Cursor, Claude Code, Cline, OpenCode, Continue.dev, Roo Code, Kilo Code, Windsurf, Void AI, Codex CLI, Antigravity CLI, and Grok Build all support MCP. Aider does not yet, though community Aider-MCP bridges exist. By the close of Q2 2026, MCP had roughly 9,400 published servers across the four major registries and ~1,300 production-ready servers, with the ecosystem mid-transition from stdio to hosted HTTP transport and from API keys to OAuth 2.1. MCP is governed by the Linux Foundation's Agentic AI Foundation (AAIF) since December 2025. ### Is Windsurf still independent? No. Codeium's Windsurf was acquired by Cognition (the Devin team). The product still ships under the Windsurf brand, with Cascade as the agent and SWE-1.5 as the in-house model. ### What happened to Void AI? The Void team announced an active-development pause in early 2026. The binary, Ollama integration, and cloud connectors all still function. Treat it as a stable open-source artifact, not a product with a roadmap. ### Cursor vs Windsurf — which IDE is better in 2026? Cursor still has tighter inline-edit ergonomics and a larger plugin ecosystem; Windsurf's Cascade plus Codemaps is better at navigating unfamiliar repos. At $15/mo Pro, Windsurf is also $5 cheaper than Cursor's entry tier, but Cursor's free Hobby plan is more generous for trial use. ### Should I use a frontier closed model or a local open one? For senior engineers shipping production code: Claude Opus 4.7 or GPT-5.5 still outperform any local model on multi-file refactors. For privacy-bound work or routine boilerplate, DeepSeek V4, DeepSeek V4 Flash, or Qwen 3.5 on Ollama are entirely viable. The honest answer is hybrid: a frontier model in the planner/architect slot and a fast local model in the editor slot. ### Which agent has the best git integration? Aider — it auto-commits each change with a sensible message and is designed to work entirely through git diffs. Claude Code is a close second with its `/commit` protocol and PR tooling. The 2026 entrants — Cursor 3's parallel agents in worktrees, Grok Build's 8 sub-agents each in isolated worktrees, and Antigravity CLI's multi-agent orchestration — all use git worktrees as the isolation primitive, which is the most under-discussed shift of the year. ### What is Cursor Composer 2.5 and how does it compare to Opus 4.7? Composer 2.5 (launched May 18, 2026) is Cursor's second-generation in-house coding model. On Artificial Analysis's Coding Agent Index it sits at 62 (third place), behind Claude Opus 4.7 in Claude Code at 66 and GPT-5.5 in Codex at 65. The headline is cost: Composer 2.5 is priced at $0.07/task (standard) or $0.44/task (Fast), versus roughly 10–60× more for the variants above it. It scored 79.8% on SWE-Bench Multilingual and 63.2% on CursorBench v3.1, and improved +35 points on SWE-Bench-Pro-Hard-AA versus Composer 2. For most agentic work it's the new value sweet spot; reach for Opus 4.7 or GPT-5.5 only when the task specifically needs frontier reasoning. ### What is Codex CLI Goal Mode (/goal) and when did it go GA? Goal Mode reached GA on May 21, 2026 in Codex CLI 0.133.0 (and matching IDE/desktop builds). You type `/goal` followed by your prompt; Codex stores the directive as a persisted thread-level state machine and resumes work after network drops, deliberate pauses, or budget resets. The same May 2026 release built on the earlier Rust rewrite of Codex CLI (versions 0.128–0.133 are all Rust). Documented six-hour runs have survived five-hour pauses without losing context. ### Should I migrate from Gemini CLI to Antigravity CLI? Yes, if you're on a free Google AI Pro or Ultra plan — Gemini CLI and Gemini Code Assist IDE extensions stop serving requests for those tiers on **June 18, 2026**. Antigravity CLI (GA May 19, 2026) is the official replacement, built in Go, and shares its agent harness with Antigravity 2.0 desktop. It supports agent skills, hooks, sub-agents, and extensions at launch. The trade-off: Antigravity CLI is not open source, unlike Gemini CLI. Paid Gemini API key access and Gemini Code Assist Standard/Enterprise licences keep access to the older CLI past the deadline. ### What is Grok Build and who is it for? Grok Build is xAI's terminal coding agent, launched May 14, 2026 in early beta. It runs up to 8 parallel sub-agents, each in its own isolated git worktree, across a three-stage plan/search/build workflow with 256K context. It scored 70.8% on SWE-Bench Verified at launch. API pricing is aggressive at $0.20/$1.50 per million tokens (input/output). Initially gated to SuperGrok Heavy ($300/mo), access expanded May 24 to SuperGrok ($30/mo) and X Premium+ ($40/mo). It's the right pick for teams that want a local-first privacy model (zero codebase data sent to xAI servers) and aggressive parallelism on hardware with the cores to handle 8 concurrent sub-agents. ### What is AGENTS.md and is it actually adopted? AGENTS.md is a simple open format for project-specific agent instructions — a README.md but for AI coding agents. Released by OpenAI in August 2025 and donated to the Linux Foundation's Agentic AI Foundation (AAIF) in December 2025, it's now adopted by **60,000+ open-source projects and agent frameworks** including Codex, Cursor, Devin, Factory, Gemini CLI, GitHub Copilot, Jules, VS Code, and Amp. If you ship one file to make your repo agent-friendly across vendors, this is the file. See our [AGENTS.md complete guide](https://codersera.com/blog/agents-md-complete-guide-2026/) for the format spec and the patterns we use internally at Codersera. ### What changed with Claude Code Skills and plugins in spring 2026? Anthropic opened the Claude Code plugin marketplace as a first-class system in spring 2026. A Skill is a single instruction set (markdown plus reference material); a plugin bundles multiple Skills, MCP servers, or commands. Both are filterable in real time from the `/skills` and `/plugin` prompts. The official marketplace is `claude-plugins-official`; the curated community list is `awesome-claude-code-plugins`. Underneath, Claude Opus 4.7's native context moved from 200K to 1M tokens, which is what makes the plugin/skill expansion practical inside a single Claude Code session. Claude Code on the web also got a sessions-sidebar redesign and drag-and-drop layout in Week 17 (April 20–24, 2026), plus custom themes and an `/ultrareview` bug-hunting preview. ### How many MCP servers exist in May 2026? Roughly 9,400 published servers across the four major registries, with about 1,300 considered production-ready. The ecosystem grew by ~1,000+ new indexed servers per month through Q1–Q2 2026 (PulseMCP's tracking), with the Q2 close projecting 14,800–22,000 servers by year-end. The structural transitions to watch in Q3/Q4 2026 are stdio → hosted HTTP transport, and API keys → OAuth 2.1. MCP itself is governed by the Linux Foundation's Agentic AI Foundation, established December 2025. ## Next steps If you've read this far you already know the choice isn't "which tool is best" but "which tool fits the codebase, the team, and the threat model." For most production teams as of May 2026 the right starter combo is Claude Code on Max-20x (Opus 4.7 + 1M-token context + plugin marketplace) plus one IDE-side agent (Cursor 3 with Composer 2.5 for cost-efficient parallel agents, or Cline 3.85 for full BYOK control), with Codex CLI `/goal` for long-horizon background runs and a local Continue.dev or OpenCode fallback for sensitive repos. Ship an AGENTS.md to your repo today; it's the cheapest single move that improves results across every vendor in this guide. The harder problem is hiring engineers who can plug an AI coding agent into a real codebase, write the MCP servers your tooling needs, and ship code that survives review. [Hire a Codersera-vetted Python or TypeScript engineer](https://codersera.com/blog/hire/hire-python-developer) who has integrated AI coding agents into production workflows. =============================================================== [11/20] AGENTS.md and SKILL.md: The Complete Guide (2026) URL: https://codersera.com/blog/agents-md-complete-guide-2026/ Last updated: 2026-05-23 =============================================================== **Quick answer.** AGENTS.md is an open, plain-Markdown file in your repo root that tells AI coding agents how to build, test, and contribute to the project. It is read natively by Codex, Cursor, Copilot, Gemini CLI, Aider, Windsurf, Zed, Factory, Jules, and over 20 other tools, is stewarded by the Linux Foundation's Agentic AI Foundation, and has been adopted by more than 60,000 repositories. Coding agents have become the default interface for a growing share of professional software work. They are also, individually, terrible at remembering project conventions across sessions. `AGENTS.md` is the format the industry settled on to fix that — a single Markdown file in the repository root that any agent can read on task start, giving it the build commands, test commands, code-style rules, and "do not touch" boundaries that would otherwise need to be re-stated in every prompt. This guide is for engineering leads picking a standard for their team, OSS maintainers deciding what to commit alongside `README.md`, and anyone who has watched an AI agent re-invent the wheel for the fourth time in a row and asked "is there a config file for this?" There is. This is it. *Last updated: May 23, 2026.* ## TL;DR - **What it is:** `AGENTS.md` is an open, vendor-neutral Markdown format placed at the project root. It contains the operational context — build commands, test commands, conventions, boundaries — that AI coding agents need but that doesn't belong in a human-facing `README.md`. - **Who reads it:** Native support in OpenAI Codex, Cursor, GitHub Copilot coding agent, Gemini CLI, Windsurf, Aider, Zed, Factory, Jules, Devin, Amp, Kilo, RooCode, Augment, Warp, JetBrains Junie, Ona, Phoenix, UiPath, Semgrep, and more. Claude Code reads `CLAUDE.md` instead, but the standard workaround is to `@AGENTS.md`-import it from `CLAUDE.md`. - **Governance:** Stewarded by the **Agentic AI Foundation under the Linux Foundation**, originally driven by OpenAI, Amp, Google (Jules), Cursor, and Factory. - **Format:** Plain Markdown. No required fields, no YAML frontmatter (yet — v1.1 proposes optional `description` and `tags`). Pick whatever H2s make sense for your repo. - **Monorepos:** Nest one per package; agents walk from the working file up to the repo root and the nearest `AGENTS.md` wins. OpenAI's own Codex monorepo ships **88** nested `AGENTS.md` files. - **Adoption:** 60,000+ repositories at time of writing; the format has become the de-facto standard since formal stewardship moved to the Linux Foundation. - **Related:** `SKILL.md` is Anthropic's separate, more structured spec for packaging reusable agent skills with YAML frontmatter and bundled scripts/references. It is complementary to `AGENTS.md`, not a competitor. ## What is AGENTS.md? `AGENTS.md` is a single, conventionally-named Markdown file placed at the root of a code repository. It contains operational context for AI coding agents: the build pipeline, the test command, the directory layout, the code-style conventions, the things that will get a junior engineer fired if they violate them. Any agent that supports the format reads it on task start and treats the contents as system-prompt-level guidance. The official spec describes it as *"a simple, open format for guiding coding agents"* and the analogy is exactly right: `AGENTS.md` is to coding agents what `README.md` is to humans. The two files coexist because they serve different audiences. A `README.md` answers "what is this project, how do I clone it, what does it do." An `AGENTS.md` answers "if you're an agent about to modify code in this repo, here is what you need to know that the README would never bother to spell out." The format is intentionally radical in its simplicity: - Plain Markdown. - No required headings. - No mandatory YAML frontmatter. - No custom syntax. - No build step. - No vendor SDK. You write Markdown. Agents read Markdown. The simplicity is the feature — the format only works as a cross-vendor standard if no vendor is incentivised to fork it. ## Why AGENTS.md matters Three structural problems forced the industry toward a single standard: ### Context windows are finite Every byte of repo context a coding agent loads competes with the byte of diff it could be writing. Without `AGENTS.md`, agents either over-read (open every file under `src/` to "understand the project") or under-read (skip the file with the test command and run the wrong one). A 200-line `AGENTS.md` at the root eliminates both failure modes. The agent loads it once, knows the test command, knows the file layout, and spends the rest of its context budget on actual code. ### Agents need to be portable between tools In 2024 every coding tool had its own config format: `.cursorrules` for Cursor, `.continuerules` for Continue, `.copilot-instructions.md` for Copilot, `CLAUDE.md` for Claude Code, custom YAML for Aider, and so on. Teams using more than one tool had to maintain N parallel files that drifted apart over time. `AGENTS.md` is the convergence point: write one file, every agent reads it, switching tools is a no-op. ### Onboarding an agent *is* onboarding The best `AGENTS.md` files read like the README of a well-run engineering team — the document you'd hand a new hire on day one. That overlap is not accidental. Anything that helps an LLM understand a repo also helps a human understand it. Teams that take `AGENTS.md` seriously end up with better-documented codebases as a side effect. ## Which coding agents support AGENTS.md The list below reflects native support — meaning the agent reads `AGENTS.md` from the repo root automatically, without a wrapper script or config flag. It is current as of May 2026. | Agent / Tool | Native AGENTS.md | File location | Fallback / notes | | --- | --- | --- | --- | | OpenAI Codex CLI | Yes | Repo root + nested + `~/.codex/AGENTS.md` | Also reads `AGENTS.override.md` for temporary overrides; 32 KiB cap. | | Cursor | Yes | Repo root | Coexists with `.cursor/rules/*.mdc`. `.cursorrules` legacy supported but deprecated. | | GitHub Copilot coding agent | Yes | Repo root | Also reads `.github/copilot-instructions.md`; AGENTS.md is the recommended cross-tool path. | | Google Gemini CLI | Yes | Repo root | Reads alongside `GEMINI.md` if present. | | Google Jules | Yes | Repo root | Original co-driver of the spec. | | Windsurf (Cognition) | Yes | Repo root | — | | Aider | Yes | Repo root | Loaded as a project-level conventions file. | | Zed | Yes | Repo root | — | | Warp | Yes | Repo root | — | | Factory CLI | Yes | Repo root + nested + `~/.factory/AGENTS.md` | Concatenates nearest-first up the tree. | | Devin (Cognition) | Yes | Repo root | — | | Amp (Sourcegraph) | Yes | Repo root | Original co-driver of the spec. | | Kilo Code | Yes | Repo root + nested | Falls back to `AGENT.md` (singular) if `AGENTS.md` absent. | | RooCode | Yes | Repo root | — | | Augment Code | Yes | Repo root | — | | JetBrains Junie | Yes | Repo root | — | | Ona | Yes | Repo root | — | | UiPath Autopilot / Coded Agents | Yes | Repo root | — | | Semgrep | Yes | Repo root | Used for AI-assisted rule writing. | | opencode / OpenCode | Yes | Repo root | — | | goose (Block) | Yes | Repo root | — | | Phoenix | Yes | Repo root | — | | VS Code Copilot custom agents | Yes | Repo root | Plus `.github/agents/*.agent.md` for per-agent profiles. | | Anthropic Claude Code | No (reads `CLAUDE.md`) | — | Workaround: `@AGENTS.md`-import inside `CLAUDE.md`. `AGENT.md` (singular) is also recognised by some installs. | The practical takeaway: if you commit one file, commit `AGENTS.md`. It is read by the broadest set of tools. If your team standardises on Claude Code, also commit a one-line `CLAUDE.md` that imports it. **Picking the right coding agent before you write AGENTS.md?** Our [**AI coding agents complete guide**](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) compares Cursor, Claude Code, Codex, Gemini CLI, Aider, Cline, Continue, and the rest head-to-head — pricing, autonomy, IDE vs terminal, and where each one wins. ## Spec deep-dive: format and (optional) frontmatter ### The shape of the file A canonical `AGENTS.md` is plain Markdown with H2 sections. There is no required ordering, no required heading set, no required heading text. The spec's only normative guidance is "use whatever headings make sense for the project." That said, the popular sections — the ones GitHub identified across 2,500+ analysed repositories and that the spec lists as recommended — are: - **Project overview** — one paragraph on what this code does and the architectural shape (monolith, monorepo, microservices, etc.). - **Setup and build commands** — every command needed to get from `git clone` to a running build. Include flags and options. - **Test commands** — how to run the test suite, how to run a single test, how to run a single suite. - **Code style guidelines** — formatter, linter, naming conventions, file-layout expectations. Cite the config file if one exists. - **Project structure** — directory tree (top 2–3 levels) with one-line descriptions per top-level folder. - **Git / PR conventions** — branch naming, commit message format, PR description template. - **Security and secrets** — what must never be committed, which paths are protected, where to put env vars. - **Boundaries** — three tiers: *always do*, *ask first*, *never do*. This is the section that prevents agent runaway. ### Optional YAML frontmatter (v1.1 proposal) The current 1.0 spec defines **no** YAML frontmatter. [Issue #135](https://github.com/agentsmd/agents.md/issues/135) proposes a 1.1 revision that adds two optional fields: ``` --- description: Short summary (<200 chars) of what this AGENTS.md governs. tags: [backend, api, python] --- ``` The rationale is *progressive disclosure*: in monorepos with dozens of nested `AGENTS.md` files, harnesses want to index them at session start without paying the token cost of reading every body. Frontmatter lets the agent see "there is an AGENTS.md at `packages/api/AGENTS.md`, it covers the backend API, it has the `python` tag" without loading the file. Status as of May 2026: **open, not merged**. Some harnesses (notably Copilot's custom-agent profiles and parts of Codex) already parse optional frontmatter for forward-compatibility; most agents simply ignore it. Don't depend on frontmatter for correctness. Do feel free to include it — it costs nothing and pays off when v1.1 ships. ### Parser behaviour Different agents parse with subtly different rules: - **Codex** concatenates root-down, joined with blank lines; skips empty files; caps total at `project_doc_max_bytes` (32 KiB default). - **Factory** walks nearest-first; closest file in the directory tree wins on conflict. - **Cursor** applies the file as rules alongside `.cursor/rules/*.mdc`; explicit rules override `AGENTS.md` guidance. - **Kilo Code** places `AGENTS.md` third in its priority stack: agent-specific prompts > project-level custom instructions > `AGENTS.md` > global custom instructions > skills. - **Claude Code** reads `CLAUDE.md` by default; the import syntax `@AGENTS.md` at the top of `CLAUDE.md` pulls the AGENTS file inline. The differences are minor enough that they don't change how you author the file — write a single, well-structured `AGENTS.md` and every tool will do something reasonable with it. ## AGENTS.md vs .cursorrules vs CLAUDE.md vs system prompts Most teams end up with two or three of these files in the same repo. The right decomposition is: | File | Read by | Use for | Don't use for | | --- | --- | --- | --- | | `AGENTS.md` | 20+ agents (universal) | Build/test commands, code style, project structure, boundaries — the things that apply regardless of which agent is editing the code | Agent-specific tool calls, model-specific tuning | | `CLAUDE.md` | Claude Code only | Claude-Code-specific workflow: subagent routing, slash commands, hook configuration, things that only Claude Code understands. Import `AGENTS.md` at the top. | Things that apply to all agents (put those in `AGENTS.md`) | | `.cursorrules` (legacy) | Cursor only (deprecated) | Nothing new. Migrate to `AGENTS.md` + `.cursor/rules/*.mdc`. | — | | `.cursor/rules/*.mdc` | Cursor only | Path-scoped or conditional rules with YAML frontmatter (`description`, `globs`, `alwaysApply`) — e.g. "when editing `**/migrations/*.py`, always include a down-migration" | Universal guidance (put that in `AGENTS.md`) | | `SKILL.md` | Claude (API, Code, claude.ai) | Reusable packaged capability with bundled scripts and references — e.g. "PDF processing skill" with `scripts/extract.py` and `references/forms.md` | Project-level conventions (those go in `AGENTS.md`) | | System prompt (in-app) | The specific agent session | One-off task guidance for a single conversation | Anything that should persist across sessions (those go in `AGENTS.md`) | Concrete rule of thumb: **if you would tell every new engineer this, it belongs in `AGENTS.md`**. If you would tell only the engineer using Claude Code, it belongs in `CLAUDE.md`. If it is a packaged, reusable capability with code, it belongs in a `SKILL.md`. If it changes every task, it stays in the prompt. ## Five real-world AGENTS.md examples worth studying The patterns below come from `AGENTS.md` files in production repositories. They are not the only valid shapes, but they are representative of what high-quality engineering teams ship. ### Example 1: The minimal canonical example From the official `agents.md` spec page. Useful as a starter for any repo: ``` # MyApp ## Dev environment tips - Use `pnpm dlx turbo run where ` to jump to a package instead of scanning with `ls`. - Run `pnpm install --filter ` to add a package to your workspace. ## Testing instructions - Find the CI plan in the `.github/workflows` folder. - Run `pnpm turbo run test --filter ` from the package root. - Always add or update tests for the code you change. ## PR instructions - Title format: `[] ` - Always run `pnpm lint` and `pnpm test` before committing. ``` ### Example 2: GitHub's spec-kit (real, large) The `github/spec-kit` `AGENTS.md` documents an "integration architecture" for adding new AI agents to a CLI toolkit. Sections include *About Spec Kit and Specify*, *Integration Architecture*, *Adding a New Integration* (a seven-step playbook), *Command File Formats* (Markdown vs TOML vs YAML), *Argument Patterns* (`$ARGUMENTS` vs `{{args}}` vs `{SCRIPT}`), *Branch Naming Convention*, and *Common Pitfalls*. The file is dense, opinionated, and treats the agent like a teammate who needs to know the team's conventions on day one. ### Example 3: Node monorepo with scoped boundaries Factory's documentation shows a Node monorepo with this shape: ``` # Acme Platform ## Stack - TypeScript 5.4, pnpm 9, Turbo 2, Vite 5, Postgres 16. ## Layout - packages/api → Express server (only edit code here for API changes) - packages/web → React + Vite (only edit code here for UI changes) - packages/shared → cross-cutting types and zod schemas ## Commands - Install: `pnpm install` - Dev: `pnpm dev` - Test: `pnpm test --filter ` - Lint: `pnpm lint` - Build: `pnpm build` ## Boundaries - Never edit `prisma/migrations/`; generate via `pnpm db:migrate`. - Never commit `.env*` files. - Ask before adding a new top-level dependency. ## PRs - Conventional commits. - Squash on merge. - Include a `Testing` section in the PR body. ``` Note the *Layout* section's "only edit code here for X" — this is one of the highest-leverage patterns. A single sentence per package eliminates the most common monorepo agent failure: cross-package edits that shouldn't exist. ### Example 4: Polyglot repo with language-specific blocks ``` # DataPlatform ## Languages This repo is Python + Go + TypeScript. Each top-level service is single-language. ### Python services - Format: `ruff format`, then `ruff check --fix`. - Type-check: `mypy --strict`. - Test: `pytest -x`. ### Go services - Format: `gofumpt -w`. - Vet: `go vet ./...`. - Test: `go test -race ./...`. ### TypeScript services - Format: `biome format --write`. - Lint: `biome check`. - Test: `vitest run`. ## Boundaries - Never call a Go service from a Python service directly; always go through the gRPC contracts in `protos/`. - Never edit generated protobuf code in `gen/`. Regenerate via `buf generate`. ``` ### Example 5: Framework-specific (Next.js) ``` # MyShop ## Stack Next.js 16 (Pages Router for marketing, App Router for /shop), Tailwind v4 in /shop only, styled-components in marketing only. ## Setup - Node 22.x, pnpm 9. - `pnpm install` - `pnpm dev` → http://localhost:3000 ## Test - `pnpm test` → vitest unit - `pnpm e2e` → Playwright - `pnpm lint:check` → ESLint - `pnpm type:check` → tsc --noEmit ## Conventions - Pages Router files: kebab-case (`product-detail.tsx`). - App Router routes: lowercase, no underscores. - Server components by default; mark client components with the `'use client'` directive at the top. - Tailwind classes only in `app/shop/**`; styled-components only in `pages/**`. ## Don't - Don't mix Tailwind and styled-components in the same file. - Don't add new Pages Router routes; new routes go in App Router under `app/shop/`. - Don't commit `.env.local` or anything in `secrets/`. ``` ## Patterns that actually change agent behaviour The reason `AGENTS.md` works is not because LLMs are great at following instructions — they aren't. It's that certain phrasings dramatically improve adherence. Patterns we've seen consistently move the needle: - **Imperative voice, not descriptive.** "Run `pnpm test` before every commit" works better than "this project uses pnpm test for testing." - **Three-tier boundaries.** Group rules into *always do*, *ask first*, and *never do*. The explicit hierarchy gives the agent a clean classification problem instead of an open-ended judgement call. - **Concrete commands with flags.** "Run `pytest -x --tb=short tests/unit`" beats "run the unit tests." - **Tech stack with version numbers.** "Node 22.x, pnpm 9, Vite 5" beats "modern JS stack." - **Real code examples.** A 5-line example of your preferred error-handling pattern is worth a paragraph of prose. - **Scope sentences per directory.** "Backend code only in `packages/api/src/`" eliminates a class of failures by itself. - **Generated-file guards.** List every directory that is regenerated by a build step (`gen/`, `__generated__/`, `migrations/`, `dist/`) and tell the agent never to edit them. ## Anti-patterns: ten things not to do 1. **Don't write a persona.** "You are a helpful AI assistant who loves clean code" wastes tokens and changes nothing. The agent already knows it is an agent. 2. **Don't paste your README.** Duplicating `README.md` content in `AGENTS.md` doubles maintenance and signals nothing new to the agent. 3. **Don't include marketing copy.** "Acme Platform is the leading developer-first observability solution" is text the agent does not need. 4. **Don't write essays.** If a section is over ~30 lines, it is probably reference documentation that belongs in `docs/` and should be linked, not inlined. 5. **Don't restate language defaults.** "Use Python 3" is implicit from `pyproject.toml`. "Use Python 3.12 with the `uv` package manager, not pip" is useful. 6. **Don't conflict with `.cursorrules` / `CLAUDE.md`.** If you keep more than one rules file, treat `AGENTS.md` as the source of truth and have the others `@import` it. 7. **Don't include secrets.** Real API keys, real DB connection strings, real internal URLs. The file is in version control; assume it leaks. 8. **Don't be vague about "good code".** "Write clean, idiomatic code" is signal-free. "Use `Result<T, E>` for fallible operations, never raise unchecked exceptions" is signal. 9. **Don't ship a 2000-line file.** Codex caps at 32 KiB. Most agents start ignoring after the first few thousand tokens. Keep it under ~500 lines; push the rest into nested per-package files or `docs/`. 10. **Don't forget to update it.** Stale `AGENTS.md` is worse than no `AGENTS.md` — the agent will confidently follow rules that no longer apply. Treat it like the README: update in the same PR as the change it documents. **Building an AGENTS.md for a Cursor-heavy team?** Read our [**Cursor IDE complete guide**](https://codersera.com/blog/cursor-ide-complete-guide-2026/) for the deeper story on how Cursor parses `AGENTS.md` alongside `.cursor/rules/*.mdc`, when path-scoped MDC rules beat a global AGENTS.md, and which Project Rules to set for a 10-engineer team. ## Monorepo patterns: nested AGENTS.md files The single biggest scale lever in the spec is nested AGENTS.md. You can put one at the repo root *and* one inside `packages/api/` *and* one inside `packages/web/`. Agents walk up the directory tree from the file they're editing and combine every `AGENTS.md` they encounter, with the closest one winning on conflicts. OpenAI's own Codex monorepo ships **88** nested `AGENTS.md` files — one per service or package. The shape is: ``` repo/ ├── AGENTS.md # global: stack, root commands, branch naming ├── packages/ │ ├── api/ │ │ ├── AGENTS.md # api: Express, Postgres conventions, test command │ │ └── src/... │ ├── web/ │ │ ├── AGENTS.md # web: React, Tailwind, Vite, component conventions │ │ └── src/... │ └── shared/ │ ├── AGENTS.md # shared: zod schemas, no DB imports allowed │ └── src/... └── tools/ └── etl/ ├── AGENTS.md # etl: Python, pandas, sqlmodel; different lint rules └── ... ``` A few practical rules: - **Defaults at the root, overrides at the leaf.** Put the stack, the branch naming, the security boundaries at the root. Put the per-package test command and the per-package conventions at the leaf. - **Don't re-state inheritance.** If the root says "pnpm 9," the leaf shouldn't say "pnpm 9" again. The agent already has the root context. - **Codex-only escape hatch:** `AGENTS.override.md` at any level *replaces* rather than extends parent instructions. Use sparingly; other tools don't honour it. - **Keep nested files short.** 30–80 lines each is the sweet spot. The root file can be longer (~200 lines). ## SKILL.md: a complementary, not competing, standard SKILL.md is Anthropic's separate spec for packaging reusable agent *capabilities*. It is more structured than AGENTS.md, with mandatory YAML frontmatter, a defined directory layout, and a three-level loading model (metadata always loaded, body loaded on trigger, bundled scripts and references loaded on demand). ### The shape of a SKILL.md ``` my-skill/ ├── SKILL.md # required: frontmatter + instructions ├── scripts/ # optional: executable code Claude runs via bash │ └── validate.py └── references/ # optional: docs loaded only when explicitly referenced └── schema.json ``` The SKILL.md itself: ``` --- name: pdf-processing description: Extract text and tables from PDF files, fill forms, merge documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction. --- # PDF Processing ## Quick start Use pdfplumber to extract text from PDFs: ```python import pdfplumber with pdfplumber.open("document.pdf") as pdf: text = pdf.pages[0].extract_text() ``` For advanced form filling, see [FORMS.md](FORMS.md). ``` ### Required frontmatter fields - `name` — lowercase, hyphens-only, max 64 characters, cannot contain "anthropic" or "claude" or XML tags. - `description` — non-empty, max 1024 characters, no XML tags. This is the *primary triggering signal*; Anthropic explicitly advises making it "pushy" about use cases to combat under-triggering. Optional: `license`. The spec is published in the open at [anthropics/skills](https://github.com/anthropics/skills) and at `agentskills.io`. ### When to use SKILL.md vs AGENTS.md | You want to… | Use | | --- | --- | | Tell every agent how to build, test, and not-break *this repo* | `AGENTS.md` | | Package a reusable capability (PDF processing, Excel generation, internal API client) with bundled scripts | `SKILL.md` | | Share project conventions across tools (Codex, Cursor, Copilot, Aider…) | `AGENTS.md` | | Ship a capability that loads only when triggered, to save context | `SKILL.md` | | Add per-package rules in a monorepo | Nested `AGENTS.md` | | Distribute capabilities to a Claude workspace, claude.ai, or Claude Code installs | `SKILL.md` | The two formats overlap in spirit (both are filesystem-resident instructions for AI agents) but solve different problems. `AGENTS.md` answers "how does this repo work?" `SKILL.md` answers "how does this reusable capability work, and when should you trigger it?" A serious team will use both: `AGENTS.md` at the repo root, and one or more `SKILL.md` bundles in `.claude/skills/` (Claude Code) or uploaded via the API. ## Migrating from .cursorrules to AGENTS.md `.cursorrules` is the legacy Cursor format. As of 2026 Cursor still reads it, but the official guidance is to migrate. The mechanical conversion is straightforward: 1. **Audit what's in `.cursorrules`.** Most files are 80% universal ("use TypeScript, format with Prettier, prefer functional components") and 20% Cursor-specific ("in Agent mode, ask before deleting files"). 2. **Move the universal 80% verbatim to `AGENTS.md`.** Same Markdown, same headings — just rename and relocate. 3. **Keep Cursor-specific rules in `.cursor/rules/*.mdc`.** The MDC format supports YAML frontmatter (`description`, `globs`, `alwaysApply`) for path-scoped rules. Use MDC for "when editing `**/migrations/*.py`, always include a down-migration" type rules; use AGENTS.md for "use Postgres parameterised queries" type rules. 4. **Delete `.cursorrules` once `AGENTS.md` is live**, or keep it as a thin pointer that says "see AGENTS.md." 5. **If you also use Claude Code,** create a one-line `CLAUDE.md` that opens with `@AGENTS.md` to import the file into Claude's context. Teams that complete this migration end up with one source of truth (`AGENTS.md`), one tool-specific surgical override location per tool, and zero drift between formats. **Comparing Claude Code, Codex, and Cursor head-to-head?** Our [**Claude Code vs OpenAI Codex deep dive**](https://codersera.com/blog/claude-code-vs-openai-codex-2026/) walks through how each one reads `AGENTS.md` in practice, where their fallback behaviours differ, and which to pick for an extended remote team. ## AGENTS.md and MCP: how they fit together A common confusion: does `AGENTS.md` replace MCP (Model Context Protocol) server configuration? No — they sit at different layers. - **`AGENTS.md`** is project-level *instructions*: what to do, what not to do, how to test, how to commit. - **MCP servers** are project-level *tools*: an HTTP/stdio endpoint the agent can call to read your database, query your monitoring system, fetch a Jira ticket, or run a custom script. A well-configured repo has both. `AGENTS.md` at the root tells the agent "run the test command before committing." A `.mcp.json` (Claude Code, Codex) or `.cursor/mcp.json` (Cursor) tells the agent *how* to call your in-house MCP servers. The instructions in `AGENTS.md` can reference MCP tools by name ("use the `db-readonly` MCP server for schema questions, never query the prod DB directly"). ## Rollout checklist for a 10-engineer team 1. **Day 1.** Audit every existing tool-specific rules file (`.cursorrules`, `.continuerules`, `.github/copilot-instructions.md`, `CLAUDE.md`). Note what's universal vs tool-specific. 2. **Day 1.** Write a 100–200 line `AGENTS.md` at the repo root covering the 8 recommended sections (overview, setup, test, style, structure, git, security, boundaries). 3. **Day 1.** Have one engineer who uses each tool (Cursor, Claude Code, Codex, Copilot) run a real task with the new file and report drift. Iterate. 4. **Day 2.** Add `@AGENTS.md` at the top of `CLAUDE.md`, plus any Claude-Code-specific subagent / slash-command guidance. 5. **Day 2.** Move Cursor-specific path-scoped rules into `.cursor/rules/*.mdc`; delete or thin `.cursorrules`. 6. **Week 1.** Add nested `AGENTS.md` files for the top 3 packages in the monorepo. 7. **Week 2.** Wire `AGENTS.md` updates into the PR template. Any structural change (new test command, new lint rule, new boundary) requires an `AGENTS.md` edit in the same PR. 8. **Month 1.** Review the rate of agent rule-violations in code review. Common patterns of violation are signals for new `AGENTS.md` sections. ## FAQ ### Is AGENTS.md officially a standard? Yes — it is stewarded by the Agentic AI Foundation under the Linux Foundation, which gives it vendor-neutral governance. The format originated from collaboration between OpenAI, Amp (Sourcegraph), Google (Jules), Cursor, and Factory, and is now adopted by more than 60,000 public repositories. ### Does Claude Code read AGENTS.md natively? Not directly. Claude Code reads `CLAUDE.md` by convention. The recommended workaround is to put `@AGENTS.md` as the first line of `CLAUDE.md` — Claude Code's `@`-import syntax pulls the contents of `AGENTS.md` into its context. Some installs also recognise the singular `AGENT.md` filename. ### Should I write AGENTS.md or .cursorrules? Write `AGENTS.md`. `.cursorrules` is Cursor-only and officially in deprecation. Anything you'd put in `.cursorrules` works in `AGENTS.md` and is portable to every other tool. ### How long should AGENTS.md be? Aim for 100–300 lines at the repo root. Codex caps at 32 KiB; other tools start ignoring after the first few thousand tokens. For larger projects, split into nested `AGENTS.md` per package rather than letting the root file grow past ~500 lines. ### Do I need YAML frontmatter? No. The 1.0 spec defines none. The 1.1 proposal (open, not merged) adds optional `description` and `tags` fields. Including them is forward-compatible and costs nothing; depending on them for correctness is not safe yet. ### AGENT.md or AGENTS.md? Always uppercase. Plural is the canonical form (`AGENTS.md`); singular (`AGENT.md`) is a fallback that some tools (Kilo, certain Claude Code installs) accept. Don't ship lowercase — case-sensitive filesystems treat `agents.md` as a different file. ### Can I put secrets in AGENTS.md? Never. The file is committed to version control. Treat it as fully public — anything you wouldn't put in `README.md` should not be in `AGENTS.md`. For secrets, document the *names* of the env vars and where to fetch their values from (Vault, AWS Secrets Manager, 1Password), not the values themselves. ### How does the monorepo precedence work exactly? Tools walk from the file being edited up to the repo root and load every `AGENTS.md` they pass. The nearest file's instructions take precedence on conflict. Codex concatenates root-down; Factory walks nearest-first; Cursor combines them with explicit MDC rules winning. The differences rarely matter as long as your nested files don't actively contradict the root. ### Is SKILL.md the same as AGENTS.md? No. `AGENTS.md` is project-level conventions read by 20+ coding agents. `SKILL.md` is Anthropic's separate spec for packaging reusable capabilities (with required YAML frontmatter, scripts, references, and a progressive-disclosure loading model) read by Claude products. Use both: `AGENTS.md` for repo conventions, `SKILL.md` for reusable capability bundles. ### Does AGENTS.md affect cost? Yes, mildly. A 200-line file is ~1–2k tokens, loaded on every agent task. The savings (fewer wrong-direction edits, fewer test-command retries, fewer "what's the lint command?" exchanges) more than offset the load cost in any non-trivial repo. Don't bloat the file past usefulness, but don't optimise for token-thrift either. ### What if my agent ignores AGENTS.md? First, check that the filename is `AGENTS.md` (uppercase, plural) at the actual working directory. Second, check that the file is under 32 KiB. Third, restart the agent session — most tools load the file on session start, not on every prompt. Fourth, check tool docs for known fallbacks (Claude Code reads `CLAUDE.md`; Gemini CLI reads both; Cursor needs MDC rules to be enabled in settings). ### Should our public OSS repo have AGENTS.md? Yes. External contributors increasingly use agents to draft PRs against your repo. An `AGENTS.md` at the root sharply improves the quality of agent-generated contributions, reduces CI cycle waste, and gives human contributors a parallel reading of "how this project actually works." It is now table-stakes for any public repo expecting external contribution. ## What this means for engineering leadership The bar for "AI-ready engineering team" has moved up. Two years ago, having any rules file at all was enough. Today, the teams getting the most out of coding agents have invested in a layered structure: a universal `AGENTS.md` at the root, nested `AGENTS.md` per package in monorepos, tool-specific overrides for the agents they standardise on, MCP server configuration alongside, and a process discipline that updates `AGENTS.md` in the same PR as the change it documents. For hiring, the signal we look for at Codersera is exactly this orchestration fluency. The engineers who deliver in 2026 don't write the most code by hand — they write the rules files, design the MCP integrations, scope the agent runs, and review the diffs critically. When we vet engineers for client teams, comfort with `AGENTS.md` authoring, monorepo nesting strategy, and the AGENTS-vs-SKILL.md-vs-MCP decomposition is now part of the technical-fit conversation, not a "nice to have." ## References and further reading - [agents.md — official spec page](https://agents.md/) - [agentsmd/agents.md — Linux Foundation repo](https://github.com/agentsmd/agents.md) - [AGENTS.md v1.1 proposal (Issue #135)](https://github.com/agentsmd/agents.md/issues/135) - [OpenAI Codex — Custom instructions with AGENTS.md](https://developers.openai.com/codex/guides/agents-md) - [Factory CLI — AGENTS.md configuration](https://docs.factory.ai/cli/configuration/agents-md) - [Kilo Code — AGENTS.md docs](https://kilo.ai/docs/customize/agents-md) - [GitHub Blog — How to write a great AGENTS.md (lessons from 2,500+ repos)](https://github.blog/ai-and-ml/github-copilot/how-to-write-a-great-agents-md-lessons-from-over-2500-repositories/) - [anthropics/skills — Anthropic SKILL.md reference repo](https://github.com/anthropics/skills) - [Anthropic — Agent Skills overview](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) - [github/spec-kit — real-world AGENTS.md example](https://github.com/github/spec-kit/blob/main/AGENTS.md) - [Codersera — Cursor IDE complete guide 2026](https://codersera.com/blog/cursor-ide-complete-guide-2026/) - [Codersera — AI coding agents complete guide 2026](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) - [Codersera — Claude Code vs OpenAI Codex (2026)](https://codersera.com/blog/claude-code-vs-openai-codex-2026/) - [Codersera — Best MCP servers for Claude Code and Cursor](https://codersera.com/blog/best-mcp-servers-claude-code-cursor-2026/) - [Codersera — Grok Build vs Claude Code vs Codex CLI](https://codersera.com/blog/grok-build-vs-claude-code-vs-codex-cli-2026/) ## Next steps If you're scaling an engineering team in 2026, the gap between "team with a working `AGENTS.md` playbook" and "team without one" shows up in code-review velocity, agent-failure rate, and onboarding time within a single sprint. Get the foundation right, and the rest of the AI-coding-agent stack (Cursor, Claude Code, Codex, Copilot, MCP, SKILL.md) compounds on top of it. If you're extending your engineering team and want senior, vetted, remote engineers who already work this way in production — engineers fluent in `AGENTS.md` authoring, monorepo nesting, MCP integration, and the agent-orchestration discipline this guide covers: [Hire a Codersera-vetted senior engineer](https://codersera.com/hire). Risk-free trial, faster hiring, lower hiring risk, and engineers who can extend your team without extending your context-switching cost. =============================================================== [12/20] Cursor IDE in 2026: The Complete Developer's Guide URL: https://codersera.com/blog/cursor-ide-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer (refreshed May 2026).** Cursor 3.5 launched on May 20, 2026 with the headline feature of **Cloud Agents** — agents that run in isolated cloud VMs with full terminal, browser, and desktop access, can work across multiple repos in parallel, and report results back to your IDE asynchronously. Cursor 3.3 (May 7) added **Build in Parallel** subagents, pinned-skill pills, Composer 2.5 with multi-file refactor at file-tree scale, and native Jira integration. Pricing tiers are unchanged from 3.2; Cloud Agents are metered separately. The competition picture has also shifted — Claude Code 2.1 with Opus 4.7 hit 80.8% on SWE-bench Verified (the highest in the category, April 2026), Cursor 3.5 is in the same ballpark for in-IDE work, and the right choice now depends more on your workflow (terminal-first vs IDE-first vs cloud-async) than on raw quality. The rest of this guide covers Cursor's core agent, Composer, model picker, team workflows, pricing, and a comparison vs Claude Code and GitHub Copilot agent mode. The May 2026 updates are summarised in the section that immediately follows. ## What changed in Cursor between March and May 2026? Three releases in eight weeks moved the product meaningfully: - **Cursor 3.3 (May 7, 2026)** — Build in Parallel: spawn N subagents from one prompt, each tackling a different sub-task, results merged. Pinned-skill pills surface frequently-used custom skills directly in the composer bar. Composer 2.5 ships multi-file refactor at file-tree scale (closer to Claude Code's batch-edit pattern). Native Jira integration: link tickets, draft PRs that auto-update issue status. - **Cursor 3.5 (May 20, 2026)** — Cloud Agents. Spin up an isolated VM, hand it a task, it runs terminal commands, browses, and edits files autonomously, with you tracking progress async in the IDE. Pinned to specific repos or repo-groups. Metered pricing on top of your existing seat. - **Plug-ins and the wider agent ecosystem** — Cursor's MCP server support, which landed late 2025, has matured; the public registry of community MCP servers crossed 200 entries by May 2026. OpenCode (the open-source coding-agent project) hit 161K GitHub stars and added a "Scout" subagent + auto-compact — worth knowing about even if you stay on Cursor. For the comparison against Claude Code, GitHub Copilot agent mode, OpenCode, and Cline as of May 2026, see our [AI coding agents complete guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/). For the rest of the Cursor surface — Composer, agent mode, model picker, pricing, team policies — read on. Cursor went from "AI-fork of VS Code" to the default editor inside many engineering orgs in roughly eighteen months. By mid-2026 it's being used by more than a million developers and ~360,000 paying customers, and it sits inside 64% of the Fortune 500. Whether you're a solo developer evaluating it for the first time, a tech lead negotiating a Teams seat count, or a CTO standardizing tooling across an extended team, the choices Cursor exposes (modes, models, rules, indexing, MCP, privacy) directly affect how your engineers ship. This guide is the version we wish we'd had when we started running Cursor in production with our own engineers and the engineers we place at client teams. It's opinionated, version-stamped, and grounded in what Cursor actually behaves like in real codebases — not just what the docs say. *Last updated: May 1, 2026.* ## TL;DR - **What it is:** Cursor is an AI-native IDE forked from VS Code, with deeply integrated agent, ask, and edit modes, codebase indexing, MCP tool support, and a model router that fronts Claude, GPT, Gemini, and DeepSeek frontier models. - **Pricing:** Six tiers — Hobby (free), Pro ($20), Pro+ ($60), Ultra ($200), Teams ($40/user), Enterprise (custom). Mid-2025 the model shifted from request quotas to a credit-based usage pool. - **Workflow:** Plan in Ask mode, implement in Agent mode, fall back to Manual edit when you want surgical control. This pattern outperforms "just talk to the agent" by a wide margin. - **Models that matter in 2026:** Claude Opus 4.7 (architecture, refactors), GPT-5.5 (general-purpose coding), DeepSeek V4 Pro (cheap reasoning), plus any OpenAI-compatible custom endpoint via your own key. - **Where it breaks:** Long agent sessions degrade reasoning quality; monorepos without `.cursorignore` hammer your indexer; multi-file refactors produce messy PRs that need human review. - **Codersera's take:** Cursor is the right default for most teams, but treat the agent as a junior engineer who needs guardrails, reviews, and a tightly scoped working set — not as an autonomous senior. ## Cursor 3.0 / 3.2: the Agents Window era Cursor 3.0 shipped on April 2, 2026 and structurally changed what "Cursor" means. The [3.0 changelog](https://cursor.com/changelog/3-0) introduces an **Agents Window** — a dedicated surface (open with `Cmd+Shift+P → Agents Window`) that "allows you to run many agents in parallel across repos and environments: locally, in worktrees, in the cloud, and on remote SSH." The IDE is still there, but for agent-heavy workflows it has been demoted to one of several panes. Coverage like [The New Stack's "Cursor's $2 billion bet: the IDE is now a fallback, not the default"](https://thenewstack.io/cursor-3-demotes-ide/) captures the framing the team is leaning into. The headline new commands are `/worktree` and `/best-of-n`, both documented in [Cursor's worktrees guide](https://cursor.com/docs/configuration/worktrees). `/worktree` tells Cursor to "do the rest of that chat in a separate checkout," so the agent's edits live in an isolated git worktree instead of mutating your main checkout. `/best-of-n` "runs the same task across multiple models at once. Each run gets its own worktree, so the candidates stay isolated from each other and from your main checkout." A real example from the docs: ``` /best-of-n sonnet,gpt,composer fix the flaky logout test ``` Three agents race the same task in three isolated worktrees; you compare diffs, pick the winner, discard the rest. Combined with the **Agent Tabs** view (multiple chats in side-by-side or grid layouts) and **Design Mode** (annotate browser UI directly to feed visual feedback back into an agent), the practical workflow shifts from "talk to one agent in one chat" to "dispatch 4–8 agents on parallel branches and adjudicate." If you used the Cursor 1.x background-agents and 2.x multitask-style flows, 3.0 is the version where this stopped being a novelty and became the primary surface. The 3.2 point release continued this trajectory with faster large-file diffs, an `Await` tool for monitoring background runs, and screenshot-based browser-automation clicking. ## 1. What Cursor actually is, and why it took off Cursor is a desktop IDE built on a fork of VS Code, with AI as a first-class primitive rather than a sidebar extension. You get the entire VS Code extension ecosystem — Prettier, ESLint, Docker, GitLens, language servers — and on top of that, three AI interaction modes (Ask, Agent, Manual), inline tab-completion, codebase-aware embeddings, MCP tool integrations, and project-scoped rules. The reason it stuck where Copilot didn't is simple: Cursor treats your repository as the unit of context, not the cursor position. Tab completion uses a fast model trained on your edits; chat and agent calls run through frontier models; both pull in semantically relevant chunks of your codebase via a vector index. The result is that the AI knows about the function three files away, not just the line you're editing. That UX advantage compounded as frontier models got better. The same Cursor session that produced mediocre output on GPT-4 in 2024 now produces production-grade refactors on Claude Opus 4.7 in 2026 — same product, different ceiling. We've covered the model side of this in [how to use Claude 4 and Sonnet with Cursor and Windsurf](https://codersera.com/blog/how-to-use-claude-4-and-sonnet-with-cursor-and-windsurf/), and the broader model landscape in our [DeepSeek V4 complete guide for 2026](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). ## 2. The three-mode workflow: Ask, Agent, Manual Cursor's interaction model is the single biggest thing engineers get wrong on day one. There are three modes, switched with `Cmd+.` or the dropdown in the chat panel, and each is right for a different kind of task. ### Ask mode Ask is read-only. It searches your codebase, answers questions, explains code, drafts plans — but it never writes a file. Use Ask to explore unfamiliar territory ("how does authentication flow through this service?"), to argue out an approach before committing to it, or to get a second opinion on a design. The official Cursor docs explicitly recommend planning in Ask before implementing in Agent, and the difference shows up in output quality. ### Agent mode Agent is the autonomous mode. You describe a task; Cursor reads files, edits multiple files, runs terminal commands, hits the web if it needs to, and iterates on errors. This is where the magic happens — and also where bad runs go off the rails. Agent mode rewards tightly scoped tasks with clear acceptance criteria ("add a rate-limit middleware to the /api/ingest route, write a Vitest test, run the tests"). It punishes vague instructions ("clean up the auth code") with sprawling, hard-to-review diffs. ### Manual edit mode Manual (formerly known as Composer) is for surgical, multi-file edits where you want the model to propose changes but not execute commands or wander off-task. It's the sweet spot when you know what you want changed and just don't want to type it. Older tutorials referencing "Composer" map to this mode. The pattern that consistently produces the best results: **scope in Ask, plan in Ask, implement in Agent, polish in Manual**. Skipping Ask is the most common cause of "Cursor wrote me garbage" complaints we hear from engineers we onboard. ## 3. Pricing in 2026 Cursor moved from request-based quotas to a credit-based pool in mid-2025. Each frontier-model call deducts from your monthly credit balance based on the model and request size. Heavy Opus 4.7 users burn credits faster than DeepSeek V4 users for the same task; the new system rewards model-routing discipline. | Plan | Price (USD/mo) | Best for | Key entitlements | | --- | --- | --- | --- | | Hobby | Free | Evaluation, side projects | Limited completions and agent requests; no MCP credits | | Pro | $20 (or $16 annual) | Individual professionals | $20 credit pool, frontier models, MCP, cloud agents, unlimited Tab | | Pro+ | $60 | Heavy daily users | 3x usage credits over Pro | | Ultra | $200 | Power users, indie founders | 20x credits across OpenAI/Anthropic/Google models, priority access to new features | | Teams | $40/user | Engineering teams (5–200) | Centralized billing, shared rules, org-wide privacy mode, RBAC, SAML/OIDC SSO, usage analytics | | Enterprise | Custom | Regulated, large orgs | Pooled usage, SCIM, audit logs, AI code tracking API, granular admin/model controls, SLA, dedicated AM | For most professional engineers Pro is enough; for full-time AI-driven development Pro+ or Ultra tend to pay back in saved credit anxiety. Teams becomes worth it as soon as you have shared rules, want one bill, or care about org-wide privacy enforcement. ### The 2025 transition to a credit pool — and the backlash Cursor's pricing structure today is the result of a contentious 2025 reset. [Vantage's pricing breakdown](https://www.vantage.sh/blog/cursor-pricing-explained) documents that "in June 2025, Cursor overhauled its pricing model, replacing fixed 'fast request' allotments with usage-based credit pools tied to actual API costs." The economic case was real — frontier-model usage was running away from any flat per-request structure — but, per the same write-up, "the rollout was rocky. Poor communication and unexpected charges led to a lot of community backlash," and Cursor "issued a public apology on July 4, 2025, offering refunds for unexpected charges incurred between mid-June and early July." The lasting effect is that the same $20 Pro that used to feel "all-you-can-eat" now visibly drains during heavy Opus 4.7 sessions. The current [Cursor pricing page](https://cursor.com/pricing) formalizes the answer to that complaint: Pro+ ($60) advertises "3x usage on all OpenAI, Claude, Gemini models" and Ultra ($200) advertises "20x usage" plus "priority access to new features." Teams ($40/user) adds shared chats, commands, rules, centralized billing, RBAC, SAML/OIDC SSO and usage analytics. Pricing-page text and per-credit costs do change — re-check before quoting numbers in a procurement deck. ## 4. Models in Cursor: built-in and custom Cursor's model picker exposes a curated list of frontier models that Cursor proxies on your behalf. As of May 2026 the most-used picks are: - **Claude Opus 4.7** — strongest model for architecture, deep refactors, and debugging gnarly logic. Higher credit cost; worth it on hard problems. - **GPT-5.5** — broad strength across coding tasks, fast, the safest "default." - **DeepSeek V4 Pro** — reasoning quality close to the frontier at roughly a tenth of the cost; great for high-volume work where you're paying yourself back per credit. See our [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) for details. - **Gemini 2.5 Pro** — long-context champion (1M+ tokens) for whole-repo passes. ### Custom OpenAI-compatible endpoints Cursor lets you bring any OpenAI-compatible API into the IDE: enter a name, base URL, and key, and the model appears in the picker marked with a person icon. Charges go directly to your provider account, bypassing Cursor's credit pool. Common reasons to do this: - You already have a self-hosted DeepSeek or Llama endpoint on your infra. - You want to point at OpenRouter or Together for a model Cursor hasn't curated yet. - You're running a privacy-sensitive setup and want every request to hit your VPC. Engineers we place often use this for cost-control pairings: a cheap custom endpoint for autocomplete-class tasks, frontier Cursor models reserved for agent runs. ## 5. Cursor Rules and `.cursorrules` Rules are the single biggest lever to make Cursor stop hallucinating in your codebase. The legacy `.cursorrules` file at repo root is now superseded by Project Rules: `.mdc` files inside `.cursor/rules/`, version-controlled per project, and scoped to specific globs. What good rules look like in 2026: - Reference canonical files instead of inlining patterns ("see `src/api/users/route.ts` for our handler shape"). - Explicit guardrails: never delete `.env` or `package.json` without confirmation; never commit without review; never assume a package exists without running `npm list <name>` first. - Verifiable goals: which lint, type-check, and test commands the agent must run before declaring done. - Tone and verbosity ("be concise; don't explain standard patterns"). - Stop conditions ("if you find a security issue, halt and report"). Teams that go from "30% of suggestions accepted" to "80%+" almost always made the jump by writing a focused rules file, not by upgrading models. Start small, add a rule when you watch the agent make the same mistake twice. ## 6. Codebase indexing, embeddings, and privacy mode When you open a repo, Cursor chunks files locally, computes a Merkle tree of file hashes, and syncs those hashes to its server. Embeddings are produced (OpenAI's embedding API or a custom embedder), stored in Turbopuffer (a remote vector DB), and used to retrieve semantically relevant chunks at query time. Re-indexing the same repo is fast because chunks are cached by hash. To preserve privacy without breaking path-based filtering, Cursor obfuscates file paths: each segment is split on `/` and `.` and encrypted with a client-held secret key. Even with the index "in the cloud," your folder structure isn't sitting plaintext in a vendor database. ### Privacy mode With Privacy Mode on, Cursor's backend doesn't retain code or data after fulfilling a request. No plaintext code is persisted server-side or in Turbopuffer; plaintext is fetched only at inference time, only for the specific files and lines a request needs, and discarded. Cursor maintains zero-data-retention agreements with all model providers it proxies. For regulated work, Privacy Mode is the floor; Enterprise adds enforced-org-wide and audit logging on top. If even Privacy Mode is too much trust to grant, you're outside Cursor's threat model and should be looking at fully-local alternatives — see our comparisons of [Cursor vs Void AI](https://codersera.com/blog/cursor-ai-vs-void-ai-an-in-depth-comparison-of-modern-ai-code-editors/) and [features, privacy, local models, and limitations in 2026](https://codersera.com/blog/void-ai-vs-cursor-features-privacy-local-models-limitations-2026/). ## 7. MCP support: connecting Cursor to your tools Model Context Protocol (MCP) is the standard for plugging external tools and data sources into AI clients. Cursor's MCP support has matured from "power-user toy" in 2024 to "first-class feature with one-click install" in 2026. How it shows up in practice: - **Three transport types:** stdio (simplest, local), SSE, and Streamable HTTP. Use stdio for CLI-shaped servers; HTTP for hosted ones. - **Two scopes:** global (Cursor Settings → Tools & MCP) or per-project via `.cursor/mcp.json` committed alongside your code. - **One-click installs** from a curated catalog (GitHub, Linear, Sentry, Postgres, Notion, Stripe, etc.) with OAuth flows handled inside Cursor. - **40-tool ceiling** per session — exceed it and Cursor stops exposing additional MCP tools to the model. Curate. The combination that consistently delivers the most value: a GitHub MCP server, a database MCP server (Postgres or your warehouse), and a docs-search server pointed at your internal Confluence/Notion. With those three, Cursor's agent stops needing to ask you for context it can fetch itself. ## MCP in 2026: "Add to Cursor", the 40-tool budget, servers worth wiring up The MCP onboarding flow tightened up in 2026. Per [Cursor's MCP docs](https://cursor.com/docs/mcp), you can now "click 'Add to Cursor' on a marketplace entry to install it and authenticate with OAuth" — one-click installs from the Cursor Marketplace and `cursor.directory` replace the old hand-edit-JSON dance. Configuration still lives in two scopes: `.cursor/mcp.json` in your repo for project-specific tools (commit it), and `~/.cursor/mcp.json` in your home directory for tools you want everywhere. The hard ceiling to plan around: third-party guides like [Truefoundry's MCP-in-Cursor guide](https://www.truefoundry.com/blog/mcp-servers-in-cursor-setup-configuration-and-security-guide) note that "Cursor has a ceiling of roughly 40 active tools across all your MCP servers combined." Past that limit, tool definitions blow your context budget and the agent silently loses access to later tools. Curate aggressively — disable servers you aren't actively using. The combinations engineers we place actually wire up: **GitHub** (PRs, issues, code search), **Postgres** or your warehouse (read-only by default), a docs-search server (Notion, Confluence), **Linear** or Jira for tickets, and **Playwright** when the agent needs to drive a browser. Five servers, well under 40 tools, and the agent stops asking you for context it can fetch itself. ## 8. Cursor in 2026: background agents, Bugbot, multitask The shape of Cursor changed materially in late 2025 and early 2026. Three releases worth knowing: - **Background agents (Cursor 1.0, mid-2025):** long-running cloud agents that can take a ticket, work for tens of minutes, and propose a PR. Now generally available on every paid tier. - **Bugbot (graduated to fixer, Feb 2026):** reviews PRs and, when it finds a real bug, spins up its own cloud agent, tests a fix, and proposes it directly on the PR. Resolution rate now around 80% — meaningfully ahead of competing review bots. - **Multitask / async subagents (Cursor 3.0):** the `/multitask` command farms a request out to parallel async subagents instead of queuing it. Combined with worktrees, you can run several isolated tasks across branches at once and pull whichever succeeds into the foreground. The practical shift: Cursor isn't just an editor anymore — it's an agent execution runtime that happens to have an editor attached. Pricing tiers track that shift; "credits" map to "agent-minutes" more closely every release. ## 9. Cursor vs the alternatives The honest comparison, after running each in production: | Tool | Form factor | Strongest at | Weakest at | Pricing (entry) | | --- | --- | --- | --- | --- | | Cursor | Forked VS Code IDE | Daily IDE workflow, Tab completion, MCP, team controls | Massive monorepo indexing, opinionated review surfaces | $20/mo Pro | | Claude Code | Terminal-based agent | SWE-bench-grade refactors, large-context reasoning, security audits | No IDE UX, no Tab autocomplete | $20/mo (Max tiers from $100) | | Windsurf | Forked VS Code IDE | Cascade context persistence, budget-friendly | Smaller MCP ecosystem; March 2026 price hike to $20 | $20/mo | | Cline | VS Code extension (open source) | Free, BYO model, transparent agent loop | You manage your own keys, ceilings, and prompts | Free | | Void AI | Fully open-source IDE | Local models, full data sovereignty | Smaller ecosystem; UX still maturing | Free | | Continue.dev | VS Code/JetBrains extension | Open-source, configurable, BYO model | Less polished agent flow than Cursor | Free | | Aider | CLI | Git-aware pair programming, scriptable workflows | No IDE; not great for exploratory work | Free (BYO model keys) | The pattern we see at engineering teams that we extend with vetted Codersera engineers: Cursor for daily IDE work, Claude Code in a terminal for big refactors and audits, Cline or Void as a local fallback when a client's data-residency policy forbids cloud inference. If you want fully-local setups, our walkthroughs cover [Void AI with Ollama on macOS](https://codersera.com/blog/run-void-ai-with-ollama-on-mac-best-cursor-alternative/), [on Ubuntu](https://codersera.com/blog/run-void-ai-with-ollama-on-ubuntu-best-cursor-alternative/), and [on Windows](https://codersera.com/blog/run-void-ai-with-ollama-on-windows-cursor-alternative/). ## 10. Real workflow examples ### Adding a feature to a Next.js app 1. Open Ask. "Where does the upload pipeline currently put files, and where do we generate signed URLs?" Skim the answer; correct any wrong assumptions. 2. Still in Ask: "Draft a plan to add server-side virus scanning before we generate the signed URL. Don't write code yet." 3. Switch to Agent with the plan in context. "Implement step 1 only. Stop after the unit test passes." 4. Review the diff line-by-line. Reject anything that touched a file outside the planned scope. 5. Repeat for steps 2 and 3. ### Refactoring a sprawling React component 1. Use Manual edit mode. Select the file. Prompt: "Extract the form-state logic into a custom hook, keeping all behavior identical. Don't change props or rendering." 2. Accept hunk by hunk; let Cursor regenerate hunks you don't like. 3. Run the typecheck and tests. If they pass, commit. ### Onboarding a new engineer to an unfamiliar codebase 1. Pin a Project Rule that points to the architecture doc and the canonical handler/component examples. 2. Have the engineer use Ask mode for the first week to build a mental model. Agent mode is off-limits until they can predict what it'll do. ## 11. Known issues and gotchas - **Long-session reasoning degradation.** Agent quality drops on very long single sessions — context starts to fragment, tool call counts balloon. Restart the session at natural seams (per task, per PR). - **Monorepo indexing.** Without a tuned `.cursorignore`, indexing a 1M-file monorepo can saturate disk IO for 5–15 minutes after opening. With one, it's 10–30 seconds. Always ignore `node_modules`, build artifacts, generated code, and any package directories irrelevant to your current scope. - **Cross-package leakage.** In monorepos, the agent will happily suggest importing backend code from a frontend package if it sees both indexed. Codify package boundaries in a Project Rule. - **Big sprawling diffs.** Multi-file refactors can produce PRs that are technically correct but practically unreviewable. Force the agent to work in smaller scopes; reject "while I was at it" changes. - **Hallucinated APIs.** Even Opus 4.7 invents methods that look right. Make "verify the package and method exist before calling" a rule. - **Credit burn.** Heavy Agent runs on Opus can chew through a $20 monthly pool in a few days. Either route routine tasks to a cheaper model (DeepSeek V4 Pro, GPT-5.5 mini) or upgrade to Pro+ / Ultra. - **Performance on huge files.** Files over ~5,000 lines slow Cursor down compared to vanilla VS Code. Split or refactor before letting the agent touch them. - **MCP tool ceiling.** 40 tools per session. Past that, the model stops seeing later tools. Curate ruthlessly. ## 12. Team and enterprise controls For organizations standardizing on Cursor: - **SSO:** SAML 2.0 (Okta, Azure AD, Google Workspace, generic). Local logins can be disabled. - **Provisioning:** SCIM for user lifecycle. - **Privacy:** Enforce Privacy Mode org-wide; Cursor maintains zero-data-retention agreements with proxied model providers. - **Compliance:** SOC 2 Type 2, GDPR, CCPA. Annual penetration testing, AES-256 at rest, TLS 1.2+ in transit. - **Admin levers:** repo allow/blocklists, model allow/blocklists, MCP server allow/blocklists, agent run defaults, AI code tracking API and audit logs (Enterprise). Most procurement objections to Cursor in 2026 are about model providers, not Cursor itself — handled with Enterprise's enforced privacy and on-prem options. ## FAQ ### Is Cursor worth $20/month? For a working engineer, yes — usually within the first week. The Tab autocomplete alone, paired with one well-scoped Agent task per day, pays back the seat. The break-even is much lower than seat cost. ### Do I need to migrate from VS Code? No painful migration. Cursor imports VS Code settings, keybindings, and extensions on first launch. You can keep both installed. ### Which model should I default to? Default to GPT-5.5 or Sonnet for everyday work; reach for Opus 4.7 on architecture and gnarly debugging; route bulk Tab-style tasks to DeepSeek V4 Pro to save credits. ### Can I use my own API keys? Yes, via custom OpenAI-compatible endpoints. Charges hit your provider account directly, bypassing Cursor's credit pool. ### Does Cursor train on my code? Not with Privacy Mode enabled, and not under enforced Enterprise privacy. Cursor also has zero-data-retention agreements with the model providers it proxies. ### How does codebase indexing actually work? Files are chunked locally, hashed into a Merkle tree, and the chunks are embedded and stored in a remote vector database (Turbopuffer). Paths are obfuscated via per-segment encryption. ### What's the difference between `.cursorrules` and Project Rules? `.cursorrules` at repo root is the legacy format. Project Rules (`.cursor/rules/*.mdc`) are the current standard — version-controlled, scoped to globs, and richer. ### Can Cursor work fully offline? No. Cursor requires cloud inference for chat and agent calls. If you need local-only inference, look at [Void AI](https://codersera.com/blog/cursor-ai-vs-void-ai-an-in-depth-comparison-of-modern-ai-code-editors/), Continue.dev with a local Ollama, or Cline pointed at a local model. ### How is Cursor different from Claude Code? Cursor is an IDE. Claude Code is a terminal agent. They overlap on agent capability but are complementary in practice — Cursor for in-flight editing, Claude Code for big async tasks. ### What's the right Cursor plan for a 10-engineer team? Teams ($40/user). You get shared rules, centralized billing, org-wide privacy enforcement, RBAC, SSO, and usage analytics. Below ~5 engineers, individual Pro often pencils out cheaper. ### Does Cursor support Jupyter notebooks? Yes, via the same VS Code Jupyter extension you'd use elsewhere. Agent edits work on cells, though large notebooks suffer from the same long-file performance hit as any 5k+ line file. ### What about Cursor's CLI? Cursor's CLI lets you start agent runs from a terminal, integrate with CI, and configure MCP from the shell. It's a complement to the IDE, not a replacement. ### Is Bugbot worth turning on? For most teams, yes. The 2026 fixer-grade Bugbot resolves close to 80% of the issues it raises and learns from PR feedback over time. Treat it as a junior reviewer that needs senior oversight, not a replacement for human review. ### How do I keep Cursor from going off the rails on a big refactor? Scope tightly, plan in Ask first, write a Project Rule that lists "don't touch X, Y, Z," restart the session per task, and review every diff. Treat the agent as a junior engineer with a long memory but no judgment. ## What this means for hiring The senior-level signal in 2026 isn't "writes clean code." Tab-completion and Agent mode mean almost any working engineer can produce passable code on a known stack. The differentiator is *orchestration*: who can scope a task narrowly enough to dispatch four to eight parallel agents with `/best-of-n`, review the resulting diffs critically, reject the "while I was at it" changes, and merge the right one. That's the same skill set that makes a strong tech lead reviewing five junior PRs — Cursor 3.0 just compresses the loop to minutes instead of days. When we vet engineers for client teams at Codersera, fluency with rules files, MCP curation, agent scoping, and worktree-based parallel runs is now part of the technical-fit evaluation, not a "nice to have." ## Next steps Cursor is a force multiplier when paired with engineers who know how to drive it — and a productivity sink when handed to engineers who treat it as autocomplete on steroids. The teams getting the most out of it have invested in rules, scoping discipline, and review culture, not just seat licenses. If you're scaling an engineering team and want senior, vetted, remote engineers who already work this way in production: [Hire a Codersera-vetted TypeScript or full-stack engineer](https://codersera.com/blog/hire/hire-typescript-developer) who works with Cursor in production. Risk-free trial, faster hiring, and engineers who can extend your team without extending your hiring risk. =============================================================== [13/20] Self-Hosting LLMs in 2026: The Complete Guide URL: https://codersera.com/blog/self-hosting-llms-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== Self-hosting large language models stopped being a hobbyist exercise sometime in late 2025. By May 2026, open-weight models (DeepSeek V4, Llama 4, Qwen 3.5, Gemma 4) match or beat the closed frontier on most non-reasoning workloads, inference engines like vLLM and SGLang have become genuinely production-grade, and Blackwell-class GPUs have collapsed the cost-per-token math against API providers. The question is no longer *can* you self-host — it is whether you should, what to run on, and how to keep latency tail-flat under real traffic. This guide is the engineering brief we hand teams at Codersera before they wire a self-hosted LLM into a product roadmap. *Last updated: May 1, 2026.* ## TL;DR - **Cost crossover vs frontier APIs lands around 2M–5M tokens/day** on reserved GPU capacity over a 12-month window. Below that, the API still wins. - **SGLang now leads vLLM by ~29% throughput on H100** (16,200 vs 12,500 tok/s on standard workloads) and up to 6x on prefix-heavy RAG pipelines thanks to RadixAttention. - **B200 delivers ~3x the throughput of H200 on Llama 2 70B Interactive** in MLPerf reporting, with HBM3e at 8 TB/s vs H200's 4.8 TB/s — and roughly 3x lower $/token in FP4 serving. - **FP8 is the production-default precision in 2026.** NVFP4 is rolling out on Blackwell but calibration tooling is still maturing; FP4 is not yet a safe default. - **DeepSeek V4 Flash needs ~158 GB of VRAM** in FP4+FP8 mixed precision — fits on a single H200, comfortable on 2x H200 for production-grade KV cache headroom. - **For dev workstations, the RTX 5090 (32 GB GDDR7, 1.79 TB/s) and Mac Studio M3 Ultra (up to 512 GB unified, 800 GB/s) cover almost every model below 70B** at usable interactive speed. ## 1. Why self-host in 2026, and when not to Self-hosting earns its keep on three axes: **unit economics at volume**, **data residency**, and **latency control**. It loses on engineering overhead and on the speed at which you can iterate on a model upgrade. The honest break-even framing: if your application processes under ~50M tokens per month, almost every cost analysis we've reviewed concludes that hosted APIs are cheaper once you account for engineering time, on-call burden, and electricity. The crossover gets serious around 11 billion tokens per month — at that scale, self-hosted infrastructure with appropriate utilization comfortably undercuts GPT-4.1 / Claude 4 tier API pricing, sometimes by 4–10x. Three non-cost reasons we still see teams self-host below the crossover: - **Regulated data (healthcare, finance, defense).** The rules don't care about your token count. - **Latency-sensitive inline UX** (autocomplete, voice, real-time agents) where the 60–250 ms of round-trip API latency is the product. - **Custom adapters or fine-tunes** that simply aren't available on hosted endpoints. If none of those apply and you're under 2M tokens/day, keep using the API and revisit in six months. If one of them applies, the rest of this guide is for you. ## 2. Hardware tiers: what to actually buy or rent The hardware decision splits cleanly into **workstation** (one developer, one model, interactive use), **single-node production** (one model serving real traffic), and **multi-node fleet** (multiple models, autoscaling, redundancy). Don't conflate them — the right answer for each is wildly different. ### Workstation tier The RTX 5090 has reset the local-inference bar. With 32 GB of GDDR7 and 1.79 TB/s of bandwidth (a 78% jump over the 4090), it hits roughly 234 tok/s on a 30B MoE at short context, over 10,000 tok/s on prefill for Qwen3 8B, and well past 17,000 generation tok/s on Qwen3-8B Q4_K_M with speculative decoding tricks. For most 7B–32B work, one 5090 is enough. Two 5090s comfortably serve a 70B model in Q4 quantization and beat an A100 on $/token. The Mac Studio M3 Ultra is the surprise winner for memory-heavy work. Up to 512 GB of unified memory at 800 GB/s means a fully unquantized 70B fits trivially, and you can hold MoE models well past 100B in aggressive quantization. MLX (Apple's native framework) consistently runs 26–30% faster than Ollama on the same hardware. We cover the model-by-model tradeoffs in our deep-dive on the [best small LLMs for local hardware](https://codersera.com/blog/best-small-llms-to-run-locally-a-comprehensive-guide/) and the [DeepSeek V4 Flash local setup walkthrough](https://codersera.com/blog/run-deepseek-v4-flash-locally-full-2026-setup-guide/), both of which lean heavily on MLX for Mac users. ### Single-node production tier This is where H100 / H200 / B200 live. The H200 (141 GB HBM3e, 4.8 TB/s) gives 1.83x–2.14x the long-context throughput of an H100 across DeepSeek, Llama, and Qwen flagships. Two H200 SXM in one pod (282 GB pooled) is the current sweet spot for V4 Flash with 256K-context KV cache headroom. The B200 (192 GB HBM3e, 8 TB/s) is the new performance king — MLPerf reporting shows roughly 3x H200 throughput on Llama 2 70B Interactive and sub-3 ms latency on 8x B200 systems. SemiAnalysis InferenceX numbers put B200 at roughly 3x lower cost-per-token than H200 in FP4 serving, despite higher hourly rental. ### Hardware comparison | Tier | GPU / system | VRAM | Bandwidth | Realistic workload ceiling | Indicative $/hr (rented) | | --- | --- | --- | --- | --- | --- | | Hobbyist | RTX 4090 | 24 GB GDDR6X | 1.0 TB/s | 13B FP16, 32B Q4 | ~$0.35 | | Workstation | RTX 5090 | 32 GB GDDR7 | 1.79 TB/s | 32B FP16, 70B Q4 | ~$0.69 | | Workstation (memory-rich) | Mac Studio M3 Ultra 512 GB | 512 GB unified | 800 GB/s | 70B FP16, 120B+ Q4 MoE | n/a (capex ~$10k) | | Single-node prod | H100 SXM 80 GB | 80 GB HBM3 | 3.35 TB/s | 70B FP8, 100B+ MoE Q4 | ~$2.5–3.0 | | Single-node prod | H200 SXM 141 GB | 141 GB HBM3e | 4.8 TB/s | V4 Flash FP4+FP8 (2x), Llama 4 405B Q4 (2x) | ~$3.5–4.5 | | Frontier prod | B200 | 192 GB HBM3e | 8.0 TB/s | V4 full FP4, frontier MoE | ~$6–8 | Two warnings on the rental numbers: spot pricing varies 30–50% week to week, and several providers (Lambda, RunPod, Modal, Together, Anyscale) trade leadership on different SKUs. Always price two providers before committing. ## 3. Inference stacks: pick the engine for the workload, not the brand The inference engine matters more than the GPU once you're past the workstation tier. There are now seven serious options. They are not interchangeable. ### vLLM vLLM is the reference production engine. PagedAttention KV cache, continuous batching, OpenAI-compatible API, the broadest model support, and the largest community. v0.6.0 delivered a 2.7x throughput improvement and 5x latency reduction over earlier releases. On H100 with FlashInfer enabled, it peaks around 12,500 tok/s on standard workloads. The new `production-stack` sub-project ships a Helm chart, Prometheus metrics, KV-cache reuse via LMCache, and Grafana dashboards out of the box. If you don't have a strong reason to pick something else, pick vLLM. ### SGLang SGLang's RadixAttention beats vLLM by ~29% on standard H100 throughput (16,200 vs 12,500 tok/s) and up to 6x on prefix-heavy workloads — multi-turn chat, RAG over shared documents, few-shot prompting — where it sustains 75–95% prefix-cache hit rates. On DeepSeek V3-class MoE models, SGLang is ~3.1x faster than vLLM. The downside: smaller community, slightly less smooth Day 0 support for new model architectures, and a steeper learning curve for the structured-generation API. ### llama.cpp The C++ engine that almost every consumer-facing tool wraps. GGUF format, ggml kernels, runs on essentially anything (CUDA, Metal, Vulkan, ROCm, CPU). For single-stream inference at the workstation tier, llama.cpp is consistently the fastest option, especially with speculative decoding and the right Q4_K_M / Q5_K_M / Q6_K quantization. It does not do continuous batching well — you don't want it as your serving layer in production. ### Ollama Ollama is llama.cpp wrapped in a daemon, a CLI, an HTTP API, and a sane model registry. As of April 2026, it is the lowest-regret entry point for local inference: `ollama pull qwen2.5:32b-instruct-q4_K_M` and you have an OpenAI-compatible endpoint on localhost in 30 seconds. We use it as the runtime in our [local agent setup guide](https://codersera.com/blog/openclaw-ollama-setup-guide-run-local-ai-agents-2026/) and our [personal AI assistant walkthrough](https://codersera.com/blog/openclaw-with-ollama-run-personal-ai-assistant-local-models/). Don't use Ollama for serving real concurrent traffic — it's optimized for one-user-at-a-time, not for batched throughput. ### LM Studio The desktop GUI. Best in class on Apple Silicon thanks to first-class MLX support; the only one of these tools where double-clicking a model file works. We compare it against the alternatives in detail in our writeup on [LM Studio vs Ollama vs OpenClaw](https://codersera.com/blog/openclaw-vs-lm-studio-vs-ollama-best-local-ai-workflow-developers/). Production-irrelevant — no Docker support, desktop-only. ### MLX Apple's framework. If you're on Mac, this is what your stack should be calling under the hood. 26–30% faster than Ollama on M3 Ultra for the same model and quantization. Native FP16, BF16, INT4, and now FP8 support on M-series. ### TGI (Text Generation Inference) HuggingFace's serving stack. As of mid-2025 it moved into maintenance mode — minor bug fixes only. It still works fine for HuggingFace Inference Endpoints customers, but new deployments should default to vLLM or SGLang. TGI now multi-backends through TRT-LLM and vLLM under the hood anyway. ### Inference engine comparison | Engine | Best for | Throughput (H100, std) | Continuous batching | Prefix caching | Operational maturity | | --- | --- | --- | --- | --- | --- | | vLLM | General production serving | ~12,500 tok/s | Yes | Yes (LMCache) | High | | SGLang | RAG, multi-turn, structured output | ~16,200 tok/s | Yes | Yes (RadixAttention) | Medium-High | | TensorRT-LLM | NVIDIA-only, max FP8/FP4 perf | ~16,200 tok/s | Yes | Partial | High (NVIDIA-supported) | | llama.cpp | Single-user, broad hardware | n/a (single-stream tool) | Limited | Manual | High (community) | | Ollama | Dev environments, agents | Inherits llama.cpp | Limited | No | Medium | | LM Studio | Mac desktop | Inherits llama.cpp / MLX | No | No | Low (desktop) | | MLX | Apple Silicon production lite | n/a (Apple-only) | Limited | Yes | Medium | | TGI | HF Inference Endpoints | Comparable to vLLM | Yes | Yes | Maintenance mode | ## 4. Models: which open weights to actually run The 2026 open-weight tier is a four-horse race: **DeepSeek V4 / V4 Flash** (March–April 2026), **Qwen 3.5** (February 2026, Apache 2.0), **Gemma 4** (April 2026, Apache 2.0), and **Llama 4** (custom community license, MoE). All four families ship MoE flagships; the dense models are now the smaller variants. For a typical product team, the decision tree is simpler than the leaderboards make it look: - **Reasoning, code, complex agents:** DeepSeek V4 or V4 Flash. V4 Flash gives you ~80% of full V4 quality at ~25% of the VRAM. Our [DeepSeek V4 Flash deep dive](https://codersera.com/blog/deepseek-v4-flash-deep-dive/) and the broader [V4 pillar](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) walk through the architecture and the FP4+FP8 mixed-precision deployment story. - **General chat, RAG, balanced workloads:** Qwen 3.5 27B (dense) or 397B-A17B (MoE). Apache 2.0, no licensing pain. Edges Gemma 4 31B on MMLU Pro (86.1% vs 85.2%) and GPQA Diamond (85.5% vs 84.3%). - **Math, competitive code, structured reasoning:** Gemma 4 31B leads the pack — AIME 2026 at 89.2%, Codeforces ELO of 2150. The 26B-A4B MoE activates only 3.8B parameters per forward pass and lands 6th on Arena AI text leaderboard at 1441 ELO. Best efficiency-per-parameter in the open ecosystem. - **Long-context document workflows:** Llama 4 Scout. 109B params, 17B active, 10M-token theoretical context (1–2M practical). License caveat: not usable in apps with over 700M MAU. - **Edge / sub-16 GB hardware:** Phi-4 (14B) for math and structured reasoning — 80.4% on MATH, beats many 30–70B models. Phi-4-mini (3.8B) at Q4_K_M runs at 15–20 tok/s on an M1 Air. Mistral Small 3 (24B dense) beats Llama 3.3 70B at a third of the params and runs on a single 4090 quantized. We have a survey of these in our [small-LLM guide](https://codersera.com/blog/best-small-llms-to-run-locally-a-comprehensive-guide/). - **Massive model on tiny hardware:** oLLM-style CPU offload tricks — covered in our writeup on [running an 80 GB model on 8 GB of VRAM](https://codersera.com/blog/how-to-run-80gb-ai-model-locally-on-8gb-vram-ollm-complete-guide/). Slow, but real. ## 5. Quantization: what's safe to ship in production Four quantization regimes matter in 2026. **FP16 / BF16** — the reference. Use when VRAM allows. No accuracy concerns, slowest throughput. **FP8** — the production default on Hopper and Blackwell. Typical accuracy degradation is 0.5–2% on standard benchmarks; head-to-head, Llama-3.1-70B at FP8 scores 69.64% on MMLU-Pro vs 70.24% BF16 (a 0.6 point gap), and matches BF16 exactly on HumanEval at 39.02%. Use FP8 by default for serving on H100/H200/B200. **INT4 (AWQ, GPTQ, GGUF Q4_K_M)** — the consumer-hardware default. AWQ retains roughly 95% of base quality, GPTQ around 90%. The Marlin kernel made INT4 actually fast on modern GPUs: Marlin-AWQ hits 741 tok/s vs vanilla AWQ's 68 tok/s (10.9x speedup); Marlin-GPTQ hits 712 tok/s vs 276 tok/s (2.6x). Sources disagree on which method wins on which task — AWQ tends to win on code, GPTQ tends to win on instruction-following (IFEval). Test on your eval set before committing. **FP4 (NVFP4, MXFP4)** — the Blackwell unlock. NVIDIA TensorRT Model Optimizer with NVFP4 gives near-FP8 quality at half the memory and double the FLOPS on B200. As of May 2026, FP4 is not the production default — calibration tooling is still maturing, and accuracy varies by model and task. Treat it as a high-value bet for cost-sensitive Blackwell deployments where you can validate end-to-end on real evals. ### Quantization quick-pick | Format | Bits | Accuracy retention | Where it shines | Where to avoid | | --- | --- | --- | --- | --- | | BF16/FP16 | 16 | 100% | Reference, training | Cost-sensitive serving | | FP8 | 8 | 98–99.5% | Hopper/Blackwell production | Pre-Hopper hardware | | AWQ-INT4 (Marlin) | 4 | ~95% | Consumer GPUs, code workloads | Strict instruction-following | | GPTQ-INT4 (Marlin) | 4 | ~90% | Cheap memory savings, IFEval | Code-heavy production | | GGUF Q4_K_M | ~4.5 | ~94% | llama.cpp / Ollama / LM Studio | Multi-tenant serving | | NVFP4 / MXFP4 | 4 | ~96–98% (model-dependent) | B200 cost optimization | Anything you can't re-evaluate | ## 6. Serving patterns: dev vs prod ### Single-GPU developer pattern One model, one GPU, OpenAI-compatible endpoint on localhost. The whole stack is `vllm serve Qwen/Qwen3.5-27B-Instruct --quantization fp8` or the equivalent `ollama run` command. Use it for prototyping, evals, and integration tests. Don't ship it. ### Multi-GPU production pattern The minimum production stack as of May 2026: - 2x H200 (or 1x B200) running vLLM or SGLang in tensor-parallel mode - An L4 or L40S in front for the embedding model (cheap, fast, separate KV pressure) - Kubernetes with the vLLM `production-stack` Helm chart, or an equivalent on Modal / RunPod / Anyscale - LMCache for cross-request KV reuse — typically 30–60% effective throughput uplift on RAG-shaped traffic - A token-aware autoscaler keying off `vllm:num_requests_waiting` and KV-cache utilization, not CPU% - Speculative decoding for latency-sensitive paths: a 1B–3B draft model in front of the production model, gives 2–3x speedup on greedy decoding ### Hybrid pattern (the underrated default) Run a self-hosted instance for high-volume, predictable workloads (embeddings, classification, summary, retrieval rewrites), and route long-tail or peak-load requests to a hosted API. Most teams over-build the self-hosted side and underestimate how much spiky traffic the API absorbs cheaply. Don't be one of them. ## 7. Monitoring and observability LLM serving has its own metrics that traditional APM doesn't capture. The non-negotiables: - **End-to-end latency** (request received → final token) — your customer-facing SLO. - **Time to first token (TTFT)** — dominated by prefill. Spikes mean queue depth or KV pressure. - **Inter-token latency (ITL)** — dominated by decode. Spikes mean batch contention or memory bandwidth saturation. - **KV cache utilization** — vLLM exposes this directly. When it crosses ~85%, latency tail blows up. This is your north-star gauge. - **Queue depth (`num_requests_waiting`)** — your autoscaler signal. - **Tokens generated / second** — capacity-planning metric, not an SLO. vLLM exposes all of this on `/metrics` in Prometheus format. Pair with Grafana dashboards (the production-stack ships defaults), wire in OpenTelemetry traces for the request lifecycle, and you'll be able to debug a tail-latency regression in minutes rather than hours. ## 8. Cost crossover, in detail The math, as cleanly as we can state it for May 2026: - **Under 50M tokens/month:** APIs win. Don't self-host. - **50M–500M tokens/month:** A reserved 2x H200 node serving Qwen 3.5 or DeepSeek V4 Flash at ~70% utilization is roughly cost-neutral with frontier APIs. The win comes from latency control or data residency, not the bill. - **500M–11B tokens/month:** Self-hosting wins on cost. Plan for 30–50% savings vs frontier APIs after engineering overhead. - **11B+ tokens/month:** Self-hosting wins decisively. 4–10x savings are realistic, especially on B200 with FP4 / NVFP4. One trap to avoid: comparing self-hosting against the *frontier* API price (GPT-4.1, Claude 4) when your workload would run fine on a cheaper hosted tier (Haiku, Gemini Flash, DeepSeek API). The API providers also have a Flash tier; the crossover against *that* is much further out — closer to 200M+ tokens/day. ## 9. Known issues and sharp edges - **KV cache fragmentation under heterogeneous request lengths.** vLLM 0.6+ helps, but you can still see 20–30% throughput loss when mixing 200-token and 32K-token requests in the same batch. Solution: route by length, or pin a separate replica for long-context. - **Speculative decoding interacts badly with structured output.** If you're using grammar-constrained generation or JSON-mode, draft models often fail validation, killing the speedup. Disable speculative for those paths. - **FP8 KV cache is not free.** Some models lose noticeable quality with FP8 KV (vs FP8 weights only). Check on your evals before flipping it on. - **NCCL across vendor / driver mismatches.** Mixing GPU generations in one tensor-parallel group is a debugging nightmare. Don't. - **Tokenizer drift.** Quantized models occasionally ship with subtly different tokenizers. Compare token counts on a fixed prompt before promoting. - **Cold start on consumer hardware.** Loading a 70B Q4 model from disk to GPU can take 60–120 seconds. Pre-warm before traffic. - **Mac Studio thermal throttling under sustained load.** Real, but only matters past 30+ minute heavy generation sessions. Unlikely to bite interactive use. - **Ollama defaults to a tiny context window** (often 2k or 4k) regardless of what the model supports. Override with `OLLAMA_CONTEXT_LENGTH` or modelfile `num_ctx`. ## 10. FAQ ### How much VRAM do I actually need for a 70B model? FP16: ~140 GB plus KV cache. FP8: ~70 GB. Q4 (AWQ/GPTQ/GGUF): ~35–40 GB plus KV. With Q4 you can run a 70B comfortably on a single H100 80 GB, or on 2x RTX 5090s with tensor parallelism. ### Is DeepSeek V4 Flash actually self-hostable? Yes, on 1x H200 (141 GB) with FP4+FP8 mixed precision at ~158 GB on-disk weights — tight but viable. 2x H200 SXM (282 GB pooled) is the production-grade configuration with comfortable KV headroom for 256K context. ### Should I use vLLM or SGLang? Default to vLLM. Switch to SGLang if your workload is RAG, multi-turn chat, or anything with heavy prefix reuse — that's where the 6x throughput edge shows up. ### Is Ollama production-ready? For a single team's internal tools, yes. For multi-tenant customer-facing serving, no — it's optimized for one user at a time. Use vLLM or SGLang for that. ### What's the cheapest way to try a 405B-class model? Rent a single 8x H100 or 4x H200 node from RunPod or Lambda for an hour. Spin up vLLM with the model in FP8, hit it from your laptop, tear it down. Total cost: under $30. ### Can I really run a 70B on a Mac? Yes — comfortably on M2 Ultra / M3 Ultra with 96 GB+ unified memory. MLX or LM Studio give you 8–15 tok/s on a Q4 70B at usable interactive latency. ### How does FP8 compare to FP4 in practice? FP8 is the boring, safe production choice in 2026. FP4 (NVFP4) on Blackwell is the cost lever for high-volume serving, but plan to validate end-to-end on your eval set — calibration tooling isn't bulletproof yet. ### Do I need Kubernetes? Below ~3 nodes, no — Docker Compose or systemd plus a load balancer is fine. Above that, the autoscaling and rollout story gets painful without it. The vLLM production-stack Helm chart is the path of least resistance. ### What about AMD MI300X / MI325X? vLLM and SGLang both support ROCm, and MI300X has 192 GB of HBM3 — more than an H100. Real-world throughput trails NVIDIA on most kernels by 10–30%, but $/token is competitive on the secondary cloud market. Worth pricing if NVIDIA capacity is unavailable. ### How do I handle model upgrades without downtime? Blue-green at the deployment level: bring up the new model on a parallel replica set, shift traffic via the load balancer, drain the old set. The vLLM production-stack supports this natively. ### What's the simplest stack for a startup hitting product-market fit? Hosted API for the user-facing path, plus one self-hosted Ollama or vLLM instance on a rented 5090 or H100 for embeddings and classification. Move more inline as your token volume crosses 50M/month. ### Is fine-tuning worth doing in 2026? For most teams, no — well-prompted Qwen 3.5, Gemma 4, or DeepSeek V4 Flash will match a fine-tuned 7B on most tasks. Fine-tune when you have a domain-specific eval that pretrained models reliably fail and you have enough labeled data (10k+ high-quality examples) to move it. ### How do I prevent KV cache OOM under load? Set `--max-num-batched-tokens` and `--gpu-memory-utilization` conservatively (start at 0.85), monitor `vllm:gpu_cache_usage_perc`, and reject or queue requests when it crosses 0.85. Don't try to use 100% of VRAM — fragmentation will get you. ### What's the right way to benchmark my own deployment? Replay real traffic, not synthetic. Capture a week of production prompts (with PII scrubbed), replay against the candidate stack at 1x, 2x, and 5x rate, and measure P50/P95/P99 TTFT, ITL, and end-to-end latency. Synthetic benchmarks lie about prefix cache hit rates. ## Next steps If you're under 2M tokens/day, stay on the API and use this guide as a six-month checkpoint. If you're crossing the threshold, the highest-leverage next moves are: (1) pick one model family and one inference engine and stick with them long enough to learn their failure modes, (2) instrument `/metrics` end-to-end before you scale, and (3) run a real-traffic replay benchmark on your candidate stack before you commit to multi-node infrastructure. Most teams that successfully ship self-hosted LLMs do it with two engineers: one who has shipped production GPU infrastructure before, and one who has shipped applied ML. If you don't have both already, that's the gap to close first. [Hire a Codersera-vetted Python or ML engineer](https://codersera.com/blog/hire/hire-python-developer) who has shipped self-hosted LLMs. =============================================================== [14/20] Apple Silicon LLMs: Complete Guide to Running Models on Mac (2026) URL: https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/ Last updated: 2026-05-31 =============================================================== **Quick answer.** In May 2026, the Mac is a credible local-LLM box. Apple Silicon's unified memory means a 64 GB MacBook runs models that won't fit on a 24 GB RTX 4090, and MLX — Apple's native ML framework — has become the fastest way to run them, beating llama.cpp by 30–40% on M5 hardware. For most users the right stack is Ollama 0.19+ (which now uses MLX under the hood on Apple Silicon) for everyday chat and agent work, MLX-LM directly when you want maximum performance or to fine-tune, and LM Studio when you want a GUI. A 32 GB M-class Mac runs 30B mixture-of-experts models at ~100 tokens/sec; a 64 GB Mac runs 70B at usable speeds; multi-Mac clusters over Thunderbolt 5 now run frontier 120B+ models for sovereign teams. This guide covers what to buy, what to install, which models to download, and how to think about quantization, fine-tuning, and clustering on Apple Silicon. ## Why run LLMs locally on a Mac in 2026? Three things changed in 2024–2026 that turned the Mac from a curiosity into the default local-LLM machine for individual developers and small teams. **Unified memory matured into an unfair advantage.** On a discrete-GPU PC, the model has to fit in the GPU's VRAM — typically 8 to 24 GB on consumer hardware, and the model weights have to be copied across PCIe to get there. On Apple Silicon, the CPU and GPU share the same pool of memory at full bandwidth. A 64 GB MacBook Pro can load a 70-billion-parameter model at 4-bit quantization and start serving tokens in seconds, no copying. The 24 GB VRAM ceiling that limits a $1,600 RTX 4090 simply doesn't apply. **MLX won.** Apple's open-source MLX framework, released late 2023, hit production maturity in 2025 and pulled decisively ahead of llama.cpp's Metal backend in 2026. On the M5 generation, MLX is 30–60% faster on most workloads and 3–4× faster on prompt processing thanks to the Neural Accelerators embedded in every GPU core. The Hugging Face `mlx-community` org now hosts ~4,800 pre-converted models. Ollama, the most popular local-LLM CLI, switched its Apple Silicon backend to MLX in version 0.19 in March 2026 — a quiet but enormous performance win for anyone already using it. **The model landscape moved.** 2026 produced a wave of mixture-of-experts (MoE) open-weight models — Qwen 3.5 / 3.6 / 3.7, DeepSeek V4 Flash, Kimi K2.6, Mixtral families — that ship a small "active" parameter budget (3B–17B activated per token) on top of a large total parameter pool. They run fast on Mac because only the active experts have to be loaded into the working set, and Apple Silicon's bandwidth handles MoE routing well. The result: a 32 GB Mac running Qwen3-Coder-30B-A3B is the practical equivalent of a much bigger GPU box for most chat and code-completion workloads. None of this means a Mac replaces a multi-GPU cloud setup for serving paying customers — it doesn't. But for privacy-sensitive work, offline use, agent prototyping, learning, and the entire "I want to use a strong model without an API bill" pattern, the Mac in 2026 is the answer. ## What is the Apple Silicon advantage, exactly? Three hardware properties drive everything: **1. Unified memory architecture (UMA).** On the M-series chip, the CPU, GPU, and Neural Engine share a single pool of RAM with no copies between them. When you load a 40 GB model on a 64 GB Mac, every component can read those weights at full memory bandwidth — typically 400–500 GB/s on a Max-class chip, 800+ GB/s on Ultra. By contrast, an NVIDIA GPU has to copy weights from system RAM into VRAM across PCIe (~64 GB/s on PCIe 5.0 x16) before it can use them, and if the model doesn't fit in VRAM at all, it can't run. **2. Memory bandwidth that's actually high for the price.** Token generation in transformers is memory-bandwidth-bound; almost no compute matters compared to "how fast can I read the weights for the next token." Bandwidth ranks roughly: M-class Pro ≈ 273 GB/s, Max ≈ 500 GB/s, M5 Max ≈ 614 GB/s, M2/M3 Ultra ≈ 800 GB/s. An RTX 4090 manages 1,008 GB/s but only on the 24 GB that fits inside it. **3. Neural Accelerators on M5.** Apple's M5 chip embeds dedicated AI accelerators inside *every GPU core* (40 in M5 Max), and MLX uses them automatically. Apple's January 2026 paper showed Qwen3-14B at 4-bit running 4.06× faster on time-to-first-token and 1.19× faster on token generation versus M4 — and that's the accelerator contribution alone, before the bandwidth gain. For interactive chat and long-context agent workloads, M5 is a genuine generational step, not just a refresh. The catch: bandwidth governs decode speed, accelerators govern prefill speed. M5's 3.5× advertised "AI gain" is mostly about prefill (the model reading your prompt). Token-by-token output speed scales with bandwidth, which only improved ~15% from M4 to M5. Don't expect a 3.5× speedup on every workload — expect a dramatic speedup on time-to-first-token for long prompts, especially in agent loops. ## What is MLX and why did it win? **MLX** ([github.com/ml-explore/mlx](https://github.com/ml-explore/mlx)) is an array-computation framework for Apple Silicon, built and open-sourced by Apple in late 2023. Think NumPy + autograd + a Metal backend, with Python and Swift APIs. Around it, `mlx-lm` ([github.com/ml-explore/mlx-lm](https://github.com/ml-explore/mlx-lm)) provides LLM-specific tooling: model loading, generation, quantization, LoRA fine-tuning, and a tokenizer wrapper. MLX's design choices that mattered: a lazy computation graph (no eager-mode overhead), zero-copy weights through unified memory (no CPU↔GPU shuttling), and a function-transform model (`grad`, `vmap`, `compile`) that lets the framework fuse operations into one Metal kernel. The result is consistently faster than llama.cpp's Metal backend on the same model + quant, with the gap widening on M5 because MLX explicitly uses the Neural Accelerators that llama.cpp doesn't. Where it stood as of May 2026: `mlx` v0.31.x, `mlx-lm` v0.31.x, roughly a release every 3–4 weeks. MoE is fully supported (the kernel work landed in late 2025). Quantization tooling covers 3-bit through 8-bit plus mixed-precision modes (the `mxfp8` and `nvfp4` formats from the broader 4-bit-floating-point ecosystem). Distributed inference and training across multiple Macs via `mx.distributed` is production-shaped. The `mlx-community` Hugging Face org has ~4,810 converted models from ~4,830 community members — pick a model, append `-4bit` or `-8bit`, and it usually exists. MLX also ships a CUDA backend (~9% of the codebase is CUDA-targeted), so the framework isn't strictly Apple-only — it can run on Linux + NVIDIA. Practically, you wouldn't use MLX on NVIDIA over vLLM or TensorRT-LLM, but it broke the "MLX locks you in" framing. ## Pick your stack: MLX-LM, Ollama, LM Studio, llama.cpp, or vllm-mlx? The Mac local-LLM stack used to be a confusing five-way choice. In 2026 it's much cleaner, because most of these now share MLX under the hood: | Tool | What it is | Right when | | --- | --- | --- | | **Ollama 0.19+** | CLI + REST server, MLX backend on Apple Silicon since March 2026 | Everyday chat, agent workflows, you want a one-line install and an OpenAI-compatible API. The default for most users. | | **MLX-LM** | Apple's official Python CLI: `mlx_lm.generate`, `mlx_lm.server`, `mlx_lm.lora` | Maximum speed, fine-tuning, scripting in Python, custom inference loops. The right answer when Ollama is too opinionated. | | **LM Studio** | Desktop GUI that wraps both GGUF (llama.cpp) and MLX backends, plus a model marketplace | You want a GUI, want to browse and download models visually, want MCP server support, or are setting up a non-technical user. | | **llama.cpp / GGUF** | The cross-platform C++ reference, Metal backend on Mac | A model is brand-new and only has GGUF conversions yet, or you want truly portable inference code that runs on Mac/Linux/Windows from one binary. | | **vllm-mlx** | vLLM's API and PagedAttention, MLX as the kernel layer | You're serving multiple concurrent users / agent fleets and need batched throughput. Worse single-user speed, much better total throughput. | Independent benchmarks on an M4 Pro 64 GB running DeepSeek V3 at Q4 in May 2026 showed: Ollama 0.19+ ≈ 58 tokens/sec single-user with ~45 ms time-to-first-token (the best for interactive chat); vllm-mlx ≈ 42 tokens/sec single-user but ~1,150 tokens/sec aggregate at 32 concurrent users (the best for agent fleets); llama.cpp Metal ≈ 52 tokens/sec, the slowest of the MLX-backed group but still the broadest model support. The "Ollama is slow on Mac" critique that was true through 2025 is now mostly false — upgrade to 0.19 or later. Jan and GPT4All are also valid choices for the "fully offline desktop app, no telemetry" niche; both lag MLX-native tools on raw speed but are excellent for privacy-strict users. ## What can I actually run on my Mac? The honest buying-guide table — what runs comfortably at 4-bit (Q4) quantization, with expected tokens/sec on an M4 Max class chip via MLX. M5 numbers run ~15% faster on token generation and dramatically faster on time-to-first-token: | Unified RAM | Realistic ceiling | Representative model | Speed (M4 Max, Q4) | | --- | --- | --- | --- | | 8 GB | 3–4 B dense, small MoE | Gemma 4 E2B, Phi-3.5-mini | 60–90 tok/s | | 16 GB | 7–8 B dense | Llama 3.2 7B, Mistral 7B | 50–70 tok/s | | 24 GB | 13–14 B dense | Qwen 3.5 14B | 30–45 tok/s | | 32 GB | **30B MoE — the 2026 sweet spot** | Qwen3-Coder-30B-A3B | **~130 tok/s** | | 48 GB | 30–40 B dense | Command-R 35B | 18–25 tok/s | | 64 GB | 70 B Q4 comfortably | Llama 3.1 70B, Qwen3.5-122B-A10B | 12–15 tok/s on 70B | | 96–128 GB | 70 B bf16, 8x22B MoE | Mixtral 8x22B, DeepSeek V4 Flash (aggressive quant) | 8–15 tok/s on 70B FP16 | | 192 GB Ultra | 405 B at Q2, DeepSeek V4 at Q3 | Llama 3.1 405B Q2_K | 3–6 tok/s | | 512 GB Studio Ultra | DeepSeek V4 at usable quant | DeepSeek V4 Q4 | 15–25 tok/s | | Multi-Mac JACCL cluster | 120B+ at higher precision | DeepSeek V4 bf16, Llama 4 400B | 10–20 tok/s, scales with cluster | **The sweet spot for most developers in May 2026 is a 64 GB M4 Max or 64 GB M5 Max.** It runs the 30B-A3B MoE class at over 100 tokens/sec (faster than most cloud APIs for interactive work), runs 70B dense at acceptable speed, and leaves headroom for the context window — which matters more than people realise, because a 128 K-token context can easily eat 8–12 GB on top of the model weights. If you have an 8 GB Mac, you can still run something useful: Gemma 4 E2B, Phi-3.5-mini, or a tiny Qwen 3.5 variant. These are not GPT-5.5-replacements, but they handle classification, summarisation, light coding assistance, and structured-output tasks well, and they run completely offline. ## Which models should I download right now? The May 2026 picks, organised by what you're trying to do. All have native MLX conversions in `mlx-community`: **Best general-purpose chat model on 32 GB:** Qwen 3.6 35B-A3B MoE. Smart, fast, multilingual, instruction-following is on par with hosted Claude Sonnet for everyday tasks. **Best coding assistant on a Mac:** Qwen3-Coder-30B-A3B at 4-bit. Roughly 130 tok/s on a 64 GB M4 Pro, ~230 tok/s on M5 Max. The state of the art for offline code completion in 2026. See our [AI coding agents guide](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) for context on how it compares to Claude Code and Cursor. **Best reasoning model on 64 GB:** DeepSeek V4 Flash at Q4 (284B total / 13B active MoE). Released late April 2026 with MIT license. Punches above its weight class on math and coding benchmarks. The full DeepSeek V4 Pro (1.6T total) only fits on a 512 GB Mac Studio Ultra or a small cluster — see our [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/). **Best 70B-class workhorse on 96 GB:** Llama 4 Maverick at Q4. The most-deployed local-LLM size for serious work. Stable, well-tuned, fine-tunable. See our [Llama 4 guide](https://codersera.com/blog/llama-4-complete-guide-2026/). **Best long-context model:** Kimi K2.6 at aggressive quant on Mac Studio Ultra, or Llama 4 Scout for its 10M-token context. Kimi K2.6 leads agentic benchmarks. See our [Kimi K2.6 guide](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/). **Best tiny model for 8–16 GB Macs:** Gemma 4 E2B / E4B. Google's small-model family, multimodal, 256K context. See our [Gemma 4 guide](https://codersera.com/blog/gemma-4-complete-guide-2026/). **Best vision-capable model:** Qwen3-VL-30B-A3B. ~68 tok/s on M4 Max, 32 GB minimum. The MLX conversion handles continuous batching for vision through vllm-mlx if you need it. The full open-weights landscape, including Qwen, Llama, DeepSeek, Mistral, Gemma, and the rest, is mapped in our [open-source LLMs landscape pillar](https://codersera.com/blog/open-source-llms-landscape-2026/). The infrastructure side — cloud GPU runners, Kubernetes deployment, multi-tenant serving — is in our [self-hosting LLMs guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/). ## How should I think about quantization on Mac? Quantization compresses weights from 16-bit floats down to lower-bit representations, trading some accuracy for huge memory savings. The two formats that matter on Mac: **MLX native quants** (`-4bit`, `-8bit`, mixed precision). The default. `mlx_lm.convert --hf-path X --q-bits 4` converts any Hugging Face model. Quality at 4-bit on 7B+ models is within 1–2% of bf16 on standard benchmarks. Sub-3B models suffer noticeably at 4-bit — prefer 8-bit there. **GGUF** (Q2_K through Q8_0, plus K-quants and IQ-quants). The cross-platform standard, used by llama.cpp, Ollama (the legacy backend), and LM Studio. Q4_K_M is the safe default for 13B+ models — within ~1% of FP16 on quality. Q5_K_M for 7B and smaller. Q8_0 if you have the RAM headroom and want near-lossless. Below Q4_K_M, use importance-matrix quantization (imatrix) variants, otherwise quality drops noticeably. What to *skip* on a Mac: AWQ, GPTQ, EXL2. All three are NVIDIA tensor-core-optimised; no Mac tool supports them natively. The few "AWQ on Mac" claims you'll see online are running the underlying base weights through a conversion, not the AWQ format itself. The general rule for Mac users in 2026: **start at MLX 4-bit**. Move to 8-bit if your model is under 7B or you notice quality issues. Move to bf16 if you have memory headroom and care about long-form reasoning quality. Only reach for GGUF when the model you want hasn't been converted to MLX yet (the gap is usually a few days). ## Can I run frontier models by clustering multiple Macs? Yes, and this is the most interesting 2026 development for sovereign-AI teams. The pattern: connect 2–4 Macs over Thunderbolt 5, install MLX, run distributed inference across the cluster. A small group of Mac Studio Ultras now serves DeepSeek V4 at higher precision than any single machine could fit. The enabling technology is **JACCL** — a distributed backend Apple shipped with macOS 26.2 that runs MLX collectives over RDMA on Thunderbolt 5, hitting 50–60 Gbps with sub-50 µs latency. An order of magnitude lower latency than the previous ring-allreduce backend. Requires a fully-connected Thunderbolt topology (every Mac directly cabled to every other Mac), which limits practical cluster sizes to 4–6 nodes. The community tool that wraps this is **EXO**, which auto-discovers Macs on the local network and distributes a single model across them, transparently. Spin up two Mac Studio Ultras, point EXO at them, and you have a single-binary serving endpoint for a 400B+ MoE that wouldn't fit on either individually. When clustering is worth it: you're a small team, you need data sovereignty (the entire model and prompt history never leaves your office), you'd otherwise be on a $50K/mo OpenAI bill, and you can amortise the hardware (~$30K–$60K depending on configuration) over a year. When it's not worth it: you're a solo developer; rent an H100 on Lambda for $2.50/hour instead. ## What about Apple's own on-device LLM (Foundation Models)? In WWDC 2025, Apple shipped the **Foundation Models framework** in iOS / iPadOS / macOS 26 — a ~3 B-parameter on-device LLM with a Swift API. It powers Apple Intelligence behind the scenes. From a developer's perspective, you call a Swift function, you get tool-using LLM output, and nothing leaves the device. Strengths: free, offline, private, optimised for Apple Silicon, no install. Weaknesses: closed weights, Swift-only, fixed size, modest quality (3 B parameters tuned for on-device, not for hard reasoning). It's not a replacement for an MLX-served open-weight model — it's a complementary option when you ship a Mac/iOS app and want a default model that just works. WWDC 2026 (June 8–12) is widely expected to expand the framework — bigger on-device models, better Swift / Xcode integration, possibly a "Core AI" replacement for Core ML. If that ships, an Apple Foundation Models pillar is the natural follow-up to this guide. ## Can I fine-tune on my Mac? Yes, with caveats. MLX-LM ships LoRA, QLoRA, and DoRA fine-tuning natively, with realistic memory footprints: - **7–8 B model:** QLoRA at 4-bit fits in 7–8 GB working memory — runs on a 16 GB MacBook Pro - **13–14 B:** 14–18 GB working memory, comfortable on 32 GB - **32 B at QLoRA:** ~20–25 GB, possible on 48 GB+ - **70 B:** requires 96 GB+ unified memory; tight but works Real-world example: a Mistral 7B QLoRA run on 5,000 examples takes ~90 minutes on an M2 Max 32 GB, peak ~7 GB RAM. The unified-memory trick again — a 32 GB Mac fine-tunes models that OOM an RTX 3090's 24 GB VRAM. The tradeoff: NVIDIA still trains 2–4× faster on whatever fits on it, so for serious multi-day fine-tuning work, rent a cloud H100. The CLI is dead simple: `mlx_lm.lora --model mlx-community/Llama-3.1-8B-Instruct-4bit --train --data ./data --iters 1000`. Our [fine-tuning LLMs guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) covers the practical recipe — dataset format, LoRA hyperparameters, evaluation — in depth. ## How do I get started in five minutes? **Path A — Ollama (recommended for most users).** 1. Install: `brew install ollama` then `ollama serve &` 2. Pull a model: `ollama pull qwen3-coder:30b` 3. Chat: `ollama run qwen3-coder:30b` 4. For programmatic use, hit the OpenAI-compatible endpoint at `http://localhost:11434/v1` **Path B — MLX-LM directly (recommended for max performance and fine-tuning).** 1. Install: `pip install mlx-lm` 2. Generate: `mlx_lm.generate --model mlx-community/Qwen3-Coder-30B-A3B-4bit --prompt "Write a Python script that..."` 3. Serve an OpenAI-compatible API: `mlx_lm.server --model mlx-community/Qwen3-Coder-30B-A3B-4bit` 4. Fine-tune: `mlx_lm.lora --model <model> --train --data --iters 1000` **Path C — LM Studio (recommended for GUI users).** Download from [lmstudio.ai](https://lmstudio.ai), hit the model marketplace, filter by "MLX" or "Apple Silicon", click download, click load, chat. Includes an OpenAI-compatible local server you can flip on with one click. ## FAQ ### Is MLX faster than llama.cpp on Mac? Yes, in 2026, by 30–60% on most workloads on M4 and M5 hardware. The gap is widest on prompt processing (time-to-first-token), where MLX uses Apple's Neural Accelerators that llama.cpp doesn't. On older M1 and M2 chips the gap is smaller. For any new Mac you buy in 2026, default to MLX-backed tooling (Ollama 0.19+, MLX-LM, vllm-mlx, or LM Studio's MLX backend). ### How much RAM do I really need for serious local-LLM work? 32 GB is the entry point for genuinely useful work — it runs the 30B-A3B MoE class at ~100 tok/s, which feels like a cloud API. 64 GB is the sweet spot — adds 70B dense models and gives breathing room for context. 96–128 GB is for power users who want bf16 70B or larger MoEs. Anything below 32 GB confines you to small models that are good for narrow tasks but won't replace a hosted API. ### Should I buy a MacBook or a Mac Studio for local LLMs? MacBook Pro 14"/16" with M4 Max 64 GB if you need portability — it's the best laptop money can buy for LLM work. Mac Studio M4 Max 64 GB if you want a desk machine at lower cost. Mac Studio M2/M3 Ultra 192 GB only if you need to run 100B+ models locally; the cost-per-token is much better than buying a discrete NVIDIA setup at the same scale, but you're paying for the unified memory pool. ### Can I run Claude or GPT on my Mac? No — neither Anthropic nor OpenAI release weights, so their models can only be accessed via their respective APIs. The best *open-weight* alternatives in May 2026 are DeepSeek V4 (rivals GPT-5.5 on many benchmarks), Kimi K2.6 (rivals Claude Opus 4.6 on agentic tasks), and Qwen 3.6 / 3.5 (the strongest all-around open-weights family). For an honest comparison see our [open-source LLMs landscape pillar](https://codersera.com/blog/open-source-llms-landscape-2026/). ### What's the difference between MLX 4-bit and GGUF Q4? Both compress a 16-bit model to roughly 4 bits per weight. MLX's 4-bit format is optimised for Apple's GPU shaders and runs faster on Apple Silicon. GGUF Q4_K_M is portable across platforms. Quality is comparable for 7B+ models. Pick MLX if the model has been converted (most have); pick GGUF only when the model is too new to have an MLX variant. ### Does the M4 vs M5 difference matter for LLMs? Yes, for prefill speed. M5 is dramatically faster at processing long prompts (3–4× faster time-to-first-token in Apple's published benchmarks) because of the Neural Accelerators. Token-by-token generation is only ~15% faster because that's memory-bandwidth-bound and bandwidth only grew modestly. If you do a lot of long-context work (RAG, agent loops, code repos), M5 is worth the upgrade. If you do short chat, the difference is smaller. ### Can I use a Mac as the primary serving box for a SaaS app? Honest answer: usually no. A Mac handles 1–10 concurrent users gracefully via vllm-mlx; beyond that you want a cloud GPU. The Mac is excellent for internal tools, agent prototypes, dogfooding, and small-team deployments. For production serving at scale, see our [self-hosting guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) for the cloud/Kubernetes path. ### What about training a model from scratch on my Mac? Not realistic at meaningful scale — you'd need weeks to months of GPU time even for a small model, and a Mac Studio Ultra doesn't beat a single rented H100 for raw FLOPS. The right path for "I want to train a model" is cloud H100/H200 spot instances. See our [guide on self-training small LLMs](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) for the realistic recipe — what fits, what costs what, and when nanochat is the right starting point. ### How do I keep MLX up to date? `pip install --upgrade mlx mlx-lm` once a month is enough. Ollama updates itself when you run `brew upgrade ollama`; LM Studio prompts you in-app. The model side moves faster than the framework side — check `mlx-community` on Hugging Face weekly for new conversions. ### What's missing or coming next? WWDC 2026 (June 8–12) is the next big inflection point. Expected: Foundation Models framework v2, deeper Xcode integration ("Xcode Intelligence"), possibly a Core AI replacement for Core ML, and likely larger on-device models. The macOS 27 release in autumn 2026 will probably ship more JACCL improvements for clusters. On the open-weights side, expect Llama 5 (probably 2027 per Meta), Qwen 4, and continued MoE growth. ## Related guides - [Self-hosting LLMs in the cloud](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) — the Linux + Kubernetes + multi-GPU side of the same coin - [Fine-tuning LLMs](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) — practical recipes for LoRA / QLoRA / DoRA / MLX-LoRA / Unsloth / Axolotl - [Self-training a small LLM from scratch](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) — when pre-training a model yourself actually makes sense - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) — the full family of open-weight models in 2026 - [DeepSeek V4 complete guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) - [Qwen 3.5 complete guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) - [Llama 4 complete guide](https://codersera.com/blog/llama-4-complete-guide-2026/) - [Gemma 4 complete guide](https://codersera.com/blog/gemma-4-complete-guide-2026/) - [Kimi K2.6 complete guide](https://codersera.com/blog/kimi-k2-6-complete-guide-2026/) - [AI coding agents (Claude Code, Cursor, Copilot)](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/) =============================================================== [15/20] Fine-Tuning LLMs: The Complete 2026 Guide (LoRA, QLoRA, Unsloth, Axolotl, MLX) URL: https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer.** Fine-tuning an open-weight LLM in May 2026 is a one-command operation. Pick QLoRA (4-bit base + LoRA adapter) with rank 16 on top of Llama 3, Qwen 3, Gemma 4, or Mistral. Use **Unsloth** on a single GPU (free tier handles 7B; a rented H100 handles 70B in under three hours), **Axolotl** for multi-GPU production, **MLX-LoRA** on Mac, or **TRL** when you need raw control. Most jobs need 500–2,000 hand-curated examples in ChatML format and run in under an hour. But the most important step is upstream: *don't fine-tune for knowledge* (use RAG), *don't fine-tune for format* (use structured outputs), and don't fine-tune until prompt engineering has stopped paying off. When you do fine-tune, the right reasons are narrow style / niche reasoning / cheaper inference via distillation. This guide covers the 2026 framework landscape, the decision tree, dataset prep, hyperparameters, evaluation, and the failure modes that quietly ruin most first runs. ## Should you fine-tune at all? Fine-tuning has a reputation as the "advanced" answer to LLM problems. In 2026 it is rarely the right first move. Three patterns reliably beat fine-tuning on the most common requests: - **RAG (retrieval-augmented generation)** is the right answer when you want the model to know *more facts* — internal docs, current events, private data, customer history. Fine-tuning baked-in knowledge ages badly (the data is stale the moment you ship) and can be wrong in ways that are hard to detect. RAG keeps the knowledge in a vector store you can update independently. - **Structured outputs / tool calling** is the right answer when you want a specific format — JSON, function call, schema-validated response. Modern model APIs and open-weight chat templates support strict JSON / Pydantic schemas; you don't need a fine-tune to get reliable structure. - **Prompt engineering with in-context examples** handles most narrow tasks if you can fit 5–10 examples in the prompt. Frontier models in 2026 have 128 K–1 M context windows; you can include a small "training set" inline and skip the fine-tune entirely. The cases where fine-tuning genuinely wins: 1. **Brand voice / style.** A 500–2,000 example QLoRA on Llama 3 8B reliably teaches a model to write in your house tone, faster and cheaper than a 10 K-token system prompt. 2. **Niche reasoning style.** Legal citations, medical SOAP notes, scientific paper sections — domain conventions that prompt engineering can describe but not reliably enforce. 3. **Distillation for inference cost.** Use Claude Opus 4.7 or GPT-5.5 to generate 50–100K high-quality examples, then fine-tune Llama 3 7B or Qwen 3.6 8B to mimic the behaviour on your narrow task. You'll serve at 1/100th the cost. 4. **Tool-calling fine-tunes** where the model needs to learn a custom function schema reliably. 5. **Multilingual or rare-language work** where the base model is weak. If your use case isn't on that list, finish prompt engineering and RAG first. Then come back. ## LoRA, QLoRA, DoRA: the parameter-efficient core Every fine-tune you'll do in 2026 uses one of these adapter techniques. They freeze the base model's billions of weights and train tiny matrices on top, capturing the "task delta" without touching the foundation. **LoRA (Low-Rank Adaptation).** Updates two small matrices *A* and *B* whose product approximates a weight delta. Two hyperparameters: `r` (rank — the bottleneck width) and `alpha` (the scale factor). A LoRA adapter is typically a few hundred MB instead of the original model's tens of GB. **QLoRA.** Keeps the base model in 4-bit NF4 quantization and trains an fp16 LoRA on top. The memory savings are dramatic — a 70B model fits in ~48 GB of VRAM instead of ~140 GB. Quality is within 1–2% of full LoRA on standard benchmarks. **This is the 2026 default.** **DoRA (Weight-Decomposed LoRA).** Decomposes the weight update into magnitude and direction, then applies LoRA only to the direction. Converges faster and often matches full fine-tuning at the same rank. Frameworks default DoRA on in 2026; `use_dora=True` is a free upgrade. **Rank and alpha defaults that work:** - Style / voice tasks: `r=16, alpha=16` - General SFT: `r=32, alpha=32` - Complex multi-turn or code: `r=64, alpha=64` Target all linear layers (`q,k,v,o,gate,up,down`). The VRAM cost of targeting all-linear vs just q/v is small and the quality gain is consistent. Ignore the old `alpha=2*r` convention; Unsloth's 2026 ablations show `alpha=r` is the cleaner default. ## Which framework should I use? Pick by the shape of your hardware, not by which library is trending on X. | Framework | Right when | Wrong when | | --- | --- | --- | | **Unsloth** | Single GPU (RTX 3090/4090/5090 or one rented H100). Indie hacker / solo dev / startup. You want fastest training in the cleanest notebook. | Multi-GPU (open-source is single-GPU only; multi-GPU costs $10/mo). | | **Axolotl** | Multi-GPU, long context (sequence parallelism), CI-integrated training, you want YAML configs you can diff in PRs. | Single GPU — Unsloth is faster. | | **HF TRL + PEFT** | You need a custom training loop, custom reward model, research, or you're building your own training framework. | Hello-world fine-tunes — too much boilerplate. | | **MLX-LM (Mac)** | Apple Silicon. Up to 70B QLoRA on 96 GB Mac Studio Ultra. Quiet, no fan noise, no cloud bill. | Serious multi-day runs; NVIDIA still trains 2–4× faster on whatever fits in VRAM. | | **LLaMA-Factory** | You want a web GUI for fine-tuning, multi-model + multi-method support, no code. | Production reproducibility (the YAML/CLI path in Axolotl is cleaner). | **Unsloth** shipped MoE support in Feb 2026 with a claimed 7–12× speedup on MoE fine-tuning, runs 500+ base models including Llama 4, Qwen 3.6, Gemma 4, DeepSeek V4, and the gpt-oss family. A 70B QLoRA fits and trains in ~2.8 hours on a single H100. Open-source Unsloth is single-GPU only; multi-GPU is Unsloth Pro ($9.99/mo) — for a solo dev this is the right tradeoff. **Axolotl v0.8.x** is the production maturity point — config + `accelerate launch -m axolotl.cli.train config.yml` is a complete pipeline. Supports SFT, LoRA, QLoRA, DPO, KTO, ORPO, GRPO, full reward modelling, quantization-aware training, and recently shipped sequence parallelism for >128 K context training. Slower per-GPU than Unsloth on single-GPU runs, but multi-GPU scaling is real and the operational story is much cleaner. **HF TRL** hit v1.0 in April 2026 and unified the post-training stack: SFTTrainer, DPOTrainer, KTOTrainer, ORPOTrainer, GRPOTrainer, RewardTrainer in one library. v1.0 even pulls in Unsloth kernels for a 2× SFT speedup. Reach for it when you want full control of the loop. **MLX-LM** on Mac handles LoRA, QLoRA, and DoRA via `mlx_lm.lora --train`. Realistic ceiling: 32 GB Mac fine-tunes 7–8B comfortably; 96 GB Mac Studio Ultra fine-tunes 70B QLoRA. The unified-memory advantage means a 32 GB Mac fine-tunes models that OOM a 24 GB RTX 3090. See our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) for the full picture. ## What about hosted fine-tuning APIs? The hosted landscape changed in early 2026: **OpenAI.** Wound down fine-tuning for new models in May 2026. Fine-tuning still works on the GPT-4.1 family and o4-mini for existing customers, but no GPT-5 / 5.4 / 5.5 fine-tunes. OpenAI's bet is that prompts + tools + memory beat bespoke weights for most use cases. Use the legacy fine-tune only if you specifically need a closed-weight API model with API-grade reliability and your data volume is too small to justify any infra (<5K examples). **Anthropic.** Claude fine-tuning is *not* generally available. Older Claude models (Haiku tier) can be fine-tuned through AWS Bedrock or Google Vertex AI under a managed-service workflow. Opus 4.7, Sonnet 4.6, and the current frontier tier are **not fine-tunable** — Anthropic positions prompt caching, extended thinking, and tool use as the right knobs. If you need Claude-quality behaviour in a fine-tune in 2026, you can't have it — distil to Llama 3 or Qwen 3.6 instead. **Together AI.** Every major Llama / Mistral / Qwen size up to 405B. LoRA at ~$0.48 per million training tokens (≤16B), full fine-tune at ~$3.20 per million tokens (70–100B). Fine-tuned adapters serve at base-model inference price plus a small overhead. **The cheapest hosted path for most teams.** **Fireworks AI.** Similar pricing to Together. Strong DPO support at 2× SFT cost. Fine-tuned models serve at base price (no surcharge). The right pick when you want RLHF-style alignment without infrastructure. **Lamini.** $0.50 per million inference tokens plus $0.50 per tuning step, with $300 free credit. Niche around their "Memory Tuning" (Mixture of Memory Experts) for factual-recall use cases where hallucination is the failure mode. Most teams don't need this; Together / Fireworks is the safer default. Rule of thumb: API-hosted wins when you'll serve under 1 M tokens/day, when you don't want to manage GPUs, or when you need an SLA you can't personally provide. Local fine-tuning wins on cost above that threshold, on privacy, and on iteration speed. ## How do I prepare a fine-tuning dataset? **Format.** ChatML (`<|im_start|>role\ncontent<|im_end|>`) is the 2026 default everywhere — OpenAI, Bedrock, Vertex, every major open-weight chat model. Use it. ShareGPT (turns array with `from`/`value`) is fine for multi-turn community datasets. Alpaca (`instruction`/`input`/`output`) is the simplest format for single-turn classification or extraction. **Pick one and be consistent.** Template inconsistency is the single biggest cause of silent fine-tune failures. **Size.** 500–1,000 clean examples genuinely works for narrow tasks (classification, format conversion, brand voice). 5,000–10,000 for general SFT. **Quality beats quantity by a wide margin.** 500 hand-curated examples consistently outperform 5,000 LLM-scraped ones with format drift and quality variance. **Hygiene.** Deduplicate (MinHash or simple exact-match), length-filter, run a quick contamination check against the eval set you'll use, validate the chat template renders correctly on 3–5 random samples before launching a multi-hour run. The `datatrove` library from Hugging Face handles dedup at scale; for a small dataset, a Python one-liner over hashes is enough. **Contamination check.** If your eval data was in the base model's pretraining corpus, your "improvement" is just recall. Always hold out a fresh eval set written or collected *after* the base model's training cutoff. A 2026 Berkeley study found 8 major agent benchmarks could be exploited to near-perfect scores without solving anything, via leaked references and broken scoring. Don't trust any benchmark you didn't read the source of. ## What hyperparameters should I use? The cheat sheet that works across most chat-style QLoRA fine-tunes: - **Learning rate:** 1e-4 to 2e-4 for LoRA on 7B–14B; 5e-5 to 1e-4 for full fine-tune; 1e-5 to 5e-5 if you see catastrophic forgetting. - **Batch size:** Effective batch of 8–32 sequences. Use gradient accumulation freely — physical batch can be as low as 1 with accumulation of 16. - **Epochs:** 1–3. More than 3 on a small dataset overfits. - **Sequence length:** Set to the 95th percentile of your dataset's tokenized length. Padding to the max wastes compute. - **Warmup:** 3–10% of total steps. Cosine decay schedule. - **Optimizer:** 8-bit AdamW (saves VRAM, minimal quality cost). - **LoRA dropout:** 0.05 is a safe default; raise to 0.1 if overfitting. Track training loss *and* validation loss every N steps; stop when validation plateaus. The most common amateur mistake is "train for 3 epochs because the docs said so" — early-stop on the validation curve. ## How do I evaluate a fine-tune? Three layers, in this order: **1. General capability regression.** Run `lm-eval-harness` (EleutherAI) on MMLU / ARC / HellaSwag / GSM8K before and after to confirm you didn't break the base. A 1–3% drop on general benchmarks is normal; 10%+ is catastrophic forgetting. **2. Task-specific eval.** Write a custom eval — 100–500 held-out examples scored by exact-match (for structured outputs) or a frontier-model judge (for generative quality). The judge pattern is standard in 2026: ask Claude Opus 4.7 or GPT-5.5 to grade outputs against a rubric. Cheap, fast, and surprisingly reliable if the rubric is clear. **3. Production A/B test.** The only eval that can't be gamed. Ship to 5–10% of traffic, log outputs, compare against the base on real workloads. Anything that passes layers 1 and 2 but fails layer 3 was suffering from train/test distribution shift you didn't catch. ## Common failure modes **Catastrophic forgetting.** Fine-tune on narrow data → model loses general capability. Mitigations: lower learning rate, train fewer epochs, mix in 10–20% general-instruction data (the *regularization* approach), prefer LoRA over full fine-tune (the base stays frozen, so forgetting is bounded), or use DoRA which empirically retains better. **Overfitting tiny datasets.** 3 epochs on 500 examples and the model memorizes them. Always hold out 5–10% as validation; early-stop on plateau. **Base-model contamination.** If your eval data was in the base's pretraining corpus, your "improvement" is just recall. Hold out fresh data written after the base's cutoff. **Template drift.** Training on ChatML, inferencing on Alpaca → nonsense outputs. Always use the same chat template at inference that you trained with. `tokenizer.apply_chat_template` is your friend. **Silent format failures.** The model trains successfully but the dataset's role tags were wrong, so it learned nothing useful. *Always* render 5 random training examples with the tokenizer's chat template and eyeball them before launching the run. ## How much does fine-tuning actually cost? Realistic 2026 cost ranges on cloud spot pricing: | Model | Method | Dataset | Hardware | Wall clock | Cost | | --- | --- | --- | --- | --- | --- | | Llama 3 7B | QLoRA r=16 | 5K examples | 1× T4 (Colab free) | ~6h | $0 | | Llama 3 7B | QLoRA r=16 | 5K examples | 1× A100 40GB | ~45 min | ~$1.50 | | Qwen 3.5 14B | QLoRA r=32 | 10K examples | 1× A100 80GB | ~2h | ~$3.50 | | Llama 3 70B | QLoRA r=64 | 10K examples | 1× H100 80GB | ~3h | ~$8 | | Mistral 7B | QLoRA on Mac | 5K examples | M2 Max 32GB | ~90 min | $0 (own hardware) | | Llama 4 70B | Full LoRA | 50K examples | 8× H100 | ~12h | ~$320 | The CoreWorxLab "fine-tune Qwen 3.5 for $11 on a rented A100" YouTube clip floating around in April 2026 is real; the workflow is plausibly Unsloth's Qwen 3.5 Colab template on a single A100. Anyone telling you fine-tuning is expensive is talking about full pre-training, not LoRA. ## Quick start: three concrete recipes **Recipe A — Llama 3 7B QLoRA on free Colab, Unsloth.** ``` !pip install unsloth from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="unsloth/llama-3-8b-bnb-4bit", max_seq_length=2048, load_in_4bit=True) model = FastLanguageModel.get_peft_model( model, r=16, lora_alpha=16, use_dora=True, target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"]) # ...load dataset, train with SFTTrainer, save adapter ``` **Recipe B — Mistral 7B QLoRA on an M-series Mac, MLX-LoRA.** ``` pip install mlx-lm mlx_lm.lora \ --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \ --train \ --data ./my-jsonl-dir \ --iters 1000 \ --batch-size 4 \ --lora-layers 16 \ --save-every 200 \ --adapter-path ./adapters ``` **Recipe C — Multi-GPU production fine-tune, Axolotl.** ``` # config.yml base_model: meta-llama/Llama-3-70B adapter: qlora load_in_4bit: true sequence_len: 4096 lora_r: 64 lora_alpha: 64 lora_dropout: 0.05 lora_target_linear: true use_dora: true datasets: - path: ./train.jsonl type: chat_template num_epochs: 2 learning_rate: 1e-4 gradient_accumulation_steps: 4 micro_batch_size: 1 warmup_steps: 100 lr_scheduler: cosine optimizer: adamw_bnb_8bit # then: accelerate launch -m axolotl.cli.train config.yml ``` ## FAQ ### What's the difference between LoRA, QLoRA, and DoRA? LoRA trains tiny adapter matrices on top of a frozen base model — much cheaper than full fine-tuning. QLoRA adds 4-bit quantization of the base, cutting memory roughly 4× with negligible quality loss. DoRA decomposes the weight update into magnitude and direction, often matching full fine-tune quality at the same rank. Default in 2026: QLoRA + DoRA together. ### How much data do I need to fine-tune? 500–2,000 examples for narrow tasks like style, voice, classification, or format conversion. 5,000–10,000 for broader instruction tuning. Quality dominates quantity: 500 hand-curated examples beat 5,000 LLM-scraped ones almost every time. ### Should I fine-tune Llama 3, Qwen 3.6, Gemma 4, or Mistral? For most general tasks in 2026: Qwen 3.6 8B or Llama 3 8B. For tiny on-device deployment: Gemma 4 E2B. For coding specifically: Qwen3-Coder. For multilingual: Qwen 3.6. See our [open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) for the full comparison. ### Can I fine-tune Claude or GPT? Not on the current frontier models. OpenAI offers fine-tuning on legacy GPT-4.1 and o4-mini. Anthropic offers Claude Haiku fine-tuning through Bedrock and Vertex but not Sonnet/Opus. The 2026 reality is: if you need Claude or GPT-class quality on a fine-tune, you can't have it directly — distil their outputs to an open-weight model instead. ### What's the cheapest way to fine-tune? Colab free tier with Unsloth for 7B QLoRA — literally $0 if it fits in your patience window. Rented H100 spot on Lambda or RunPod at $2.50/hr for serious work — most 7B–14B fine-tunes finish in under an hour. Together AI's hosted fine-tuning at ~$0.48 per million training tokens if you don't want to touch GPUs at all. ### How long does a typical fine-tune take? A 7B QLoRA on 5K examples: ~45 min on an A100, ~6h on Colab free T4. A 70B QLoRA on 10K examples: ~3h on an H100. A full multi-day pre-training run is a different problem entirely — see our [self-training guide](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/). ### When is RAG better than fine-tuning? Always, when the goal is to give the model access to *knowledge*. RAG keeps data fresh, auditable, and updatable independently of the model. Fine-tuning baked-in facts is brittle: the data stales, hallucinations are harder to detect, and updating means a new training run. Use fine-tuning for *behaviour* — style, format, reasoning patterns — and RAG for *facts*. ### What about catastrophic forgetting? Real and common. The model "forgets" general capabilities while specialising. Mitigations: lower learning rate (1e-5 to 5e-5), train fewer epochs (often 1 is enough), mix 10–20% general instruction data into your training set, prefer LoRA over full fine-tune (the base stays intact under the adapter), and use DoRA which retains better empirically. ### Can I run a fine-tuned model on a Mac? Yes. Train on cloud H100 if needed, then download the adapter and merge it with the base, convert to MLX 4-bit, and run via `mlx_lm.generate` or Ollama. The whole serving side is covered in our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/). ### What about DPO, ORPO, GRPO, SimPO? Post-SFT alignment techniques for preference tuning. DPO is the foundational pairwise-preference method. ORPO merges SFT and DPO into one stage — the simplest pipeline. GRPO is what DeepSeek R1 popularised for reasoning-with-verifiable-rewards (math, code). For most fine-tunes: SFT alone is enough. Add ORPO if you have preference pairs. Reach for GRPO only if you're going after a reasoning benchmark specifically. ### How do I deploy a fine-tuned model? Three options. (1) Self-host: convert adapter to MLX or GGUF, serve via Ollama / MLX-LM / vLLM. (2) Hosted: upload the adapter to Together / Fireworks, serve at base-model price. (3) Edge: quantize to 4-bit and run on-device. The right path depends on traffic volume and SLA needs — see our [self-hosting guide](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/). ## Related guides - [Apple Silicon LLMs](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) — running and fine-tuning models on Mac with MLX - [Self-training a small LLM from scratch](https://codersera.com/blog/self-training-small-llm-complete-guide-2026/) — when pre-training a model yourself actually makes sense - [Self-hosting LLMs](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) — serving infrastructure in the cloud - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) — which base model to fine-tune - [Llama 4 guide](https://codersera.com/blog/llama-4-complete-guide-2026/) - [Qwen 3.5 guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) - [Gemma 4 guide](https://codersera.com/blog/gemma-4-complete-guide-2026/) - [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) =============================================================== [16/20] Self-Training a Small LLM From Scratch: The 2026 Complete Guide URL: https://codersera.com/blog/self-training-small-llm-complete-guide-2026/ Last updated: 2026-05-27 =============================================================== **Quick answer.** Pre-training an LLM from scratch in 2026 is dramatically cheaper than it was in 2024 — you can train a GPT-2-quality model end-to-end for around **$50 on an 8× H100 spot instance** using [Karpathy's nanochat](https://github.com/karpathy/nanochat) — but for almost every product use case the right answer is still to fine-tune an existing open-weight model, not pre-train. Pre-training makes sense when (a) you have a genuinely novel domain (rare language, niche scientific corpus, proprietary code) with 10B+ clean tokens no open model has seen, (b) you have $20K+ in compute budget you've already exhausted on fine-tuning, or (c) you want to learn how the stack works. This guide covers the realistic recipes — nanochat for learning, modded-nanoGPT for speedrunning, lit-gpt for production — the modern 2026 architecture template (decoder-only + RoPE + RMSNorm + SwiGLU + GQA), the open datasets that actually work (FineWeb, FineWeb-Edu, SlimPajama, The Stack v2), honest cost estimates across model sizes, and a decision tree for when self-training beats fine-tuning. Reading it carefully will save most teams from a six-figure mistake. ## Should you train a model from scratch? For 99% of teams, no. Pre-training a model means building from zero what Meta, OpenAI, and Anthropic spend tens of millions of dollars on every few months. Even with 2026's cost reductions, a Llama-3-class 8B model trained at the now-standard "trillions of tokens" scale costs **$1M–2M** on spot pricing. A Chinchilla-optimal 7B at 140B tokens is ~$30K on spot. A tiny "learning project" 700M model trained to GPT-2 quality is genuinely $50. The handful of cases where pre-training actually wins: - **Genuinely novel domain.** Medical, legal, code in a niche language, a rare natural language, a closed scientific corpus the open models haven't seen. If your tokens are meaningfully different from web text, an existing tokenizer will be inefficient and an existing model will hallucinate where it should defer. - **Data sovereignty.** Regulated industries where you can't use a model trained on unknown web data — defence, certain healthcare deployments, classified work. - **Edge-deployment niche.** A 150M–500M model purpose-built for your task and quantised to 4-bit can run on a phone CPU. Llama 3 1B is overkill for most edge use cases. A custom small model can be 3–10× more efficient on your domain. - **Learning.** Understanding the full stack — tokeniser, dataloader, optimiser, distributed training, eval — is the highest-leverage skill in AI engineering. [Karpathy's nanochat](https://github.com/karpathy/nanochat) exists for exactly this. It's the best 4 hours you can spend. For everything else: [fine-tune an open-weight model](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) instead. Better quality, faster, cheaper, lower risk. ## What does "small LLM" actually mean in 2026? The 2026 small-LLM bands and what they cost to train, on cloud spot pricing (H100 at ~$2.50/hr, 8× node at ~$18/hr): | Target | Tokens | GPU-hours (H100 80GB) | Spot cost | Real-world example | | --- | --- | --- | --- | --- | | 150M, 30B tokens | 30B | ~150 | ~$200 | Tiny edge model | | 700M, 40B tokens | 40B | ~30 on 8× H100 | ~$50 | nanochat speedrun | | 1B, 50B tokens | 50B | ~600 | ~$1.5K | Chinchilla-optimal 1B | | 1B, 3T tokens | 3T | ~16,000 | ~$10K–15K | TinyLlama scale (inference-optimal) | | 3B, 60B tokens | 60B | ~5,500 | ~$14K | Chinchilla-optimal 3B | | 7B, 140B tokens | 140B | ~30,000 | ~$30K–50K | Chinchilla-optimal 7B | | 7B, 15T tokens | 15T | ~3,000,000 | ~$1M–2M | Llama-3-class quality | H200 is ~1.3× faster than H100, B200 is ~2–3× — those numbers shift downward 30–60% on newer hardware. The dramatic point is that **"GPT-2 quality" is now a $50 weekend project**; the dramatic point in the *other* direction is that **Llama 3 quality is a $1M+ training run** even at spot. ## Has Chinchilla's "20 tokens per parameter" rule broken? Yes, in the inference-optimal direction. Chinchilla (2022) defined compute-optimal training: train a 1B model on ~20B tokens, a 70B on ~1.4T. In 2024–2026, the field shifted to **inference-optimal** training — train far past Chinchilla because the per-token inference savings dwarf the extra training cost when you're going to serve the model for years. Llama 3 8B trained at ~200:1 tokens-per-parameter (15T tokens for 8B). Liquid AI's LFM2.5-350M hit 80,000:1 (28T tokens on 350M params) in April 2026. The general 2026 rule for any model you'll actually serve: **train at least 100:1, ideally 500–1,000:1 for small models**. For a $50 learning project, Chinchilla is fine; for a production model you'll embed in an app, push way past. ## What's the modern 2026 architecture? The small-LLM architecture is essentially solved. Every open-weight model that matters in 2026 — Llama 3, Qwen 3.6, Gemma 4, Mistral, DeepSeek V4, Granite 4 — converged on the same template. Copy it; don't innovate: - **Decoder-only transformer.** Encoder-decoder is dead for language; sparse-attention / SSM hybrids are experimental. - **Pre-norm with RMSNorm.** LayerNorm is legacy. RMSNorm trains more stably and slightly faster. - **Rotary positional embeddings (RoPE).** With YaRN or similar scaling for context-length extension. - **SwiGLU FFN.** Gated linear unit with Swish activation. Replaced ReLU/GELU FFNs. - **Grouped-query attention (GQA).** Typically 4–8× fewer KV heads than Q heads. Cuts KV-cache memory dramatically with negligible quality loss. - **No bias terms.** In Q/K/V projections and FFN, bias-free is standard. - **Tied embeddings** for small models (output projection shares weights with input embedding). Save parameters that aren't pulling weight. If you want to experiment beyond this: MoE for parameter efficiency at the cost of training complexity, sliding-window attention for cheap long context, or Mamba/SSM hybrids for very long sequences. Otherwise: copy a Llama-3 config, scale dimensions to your budget, move on. ## Which reference recipe should I use? Three project shapes, three right answers: **1. Learning / "I want to understand the stack" —**[**karpathy/nanochat**](https://github.com/karpathy/nanochat)**.** The canonical solo-dev starting point. $48 on an 8× H100 node (~$15 on spot), ~1.65 hours wall clock as of March 2026. Single `speedrun.sh` script takes you tokenizer → pretrain → mid-train → SFT → optional RL on GSM8K → eval → web chat UI. The code is short enough to read end-to-end in an afternoon. **If you're new to pre-training, start here.** **2. Speed-record geeking —**[**KellerJordan/modded-nanogpt**](https://github.com/KellerJordan/modded-nanogpt)**.** The community speedrun fork of nanoGPT. As of April 2026, the record is *1.35 minutes to GPT-2-quality on 8× H100*, achieved through the Muon optimiser, Flash Attention 3, FP8 head, learnable cross-stream attention, and multi-token prediction. Reading the commit history is a graduate course in modern training tricks. Use this when you want to learn what's actually fast in 2026. **3. Production-grade work —**[**Lightning-AI/litgpt**](https://github.com/Lightning-AI/litgpt)**.** 20+ supported architectures, production-grade pretrain + fine-tune + deploy. Powered the original TinyLlama. Use this when nanochat's hackability stops being a feature and you need a stable CLI, FSDP2 / TP scaling, and someone-else-can-inherit-this configuration. Pair with **torchtitan** for multi-node training beyond a single 8× H100 box. Other notable references worth reading: - [EleutherAI Pythia](https://huggingface.co/EleutherAI/pythia-1.4b) — 160M to 12B series with intermediate checkpoints, the gold-standard "training dynamics" reference - [Allen AI OLMo](https://allenai.org/olmo) — fully open weights + data + intermediate states - [TinyLlama](https://github.com/jzhang38/TinyLlama) — the canonical 1.1B model trained on 3T tokens, the inference-optimal pattern ## What datasets should I use? For general pre-training, three open corpora dominate 2026: **FineWeb** (Hugging Face, 15T tokens). Common Crawl with careful filtering. The default open base. Outperforms RedPajama V1 and RefinedWeb in head-to-head ablations. **FineWeb-Edu** (1.3T tokens). Educational-content-filtered subset of FineWeb. Dramatically better MMLU / ARC numbers per training-token than vanilla FineWeb. Use when knowledge density matters for the downstream tasks. **RedPajama V2** (100T+ tokens with quality metadata). Useful when you want to apply your own filtering — the metadata lets you slice by quality, language, source domain. **SlimPajama** (627B tokens). Deduplicated subset of RedPajama V1. The right "small but clean" web corpus when your training budget is a single 8× H100 box and you don't have time to do your own dedup at scale. Domain-specific corpora that pair well: - **The Stack v2** — code, ~67T tokens, opt-out respected - **PG-19** — long-form prose, classic literature - **Wikipedia dumps** — factual grounding, multilingual - **arXiv** — scientific writing - **OpenWebMath** — mathematical reasoning - **SmolLM-Corpus** — Hugging Face's curated small-model training set with deduplication done **Deduplication is mandatory.** Duplicated examples poison training dynamics — the model overfits to whatever appeared multiple times. Use `datatrove` (Hugging Face) or `text-dedup` for scale; MinHash + LSH at large scale; exact-match for small corpora. ## Should I train my own tokenizer? Usually no. Adopt an existing tokenizer (Llama 3, Qwen, Mistral) and inherit a battle-tested vocabulary. You'll save days of iteration and avoid subtle bugs. Train your own only when you have a heavy domain skew — code-heavy, non-English, scientific notation. The test: measure your dataset's compression rate (bytes-per-token) on your data vs the Llama tokenizer. If you save >15%, you'll feel that gain forever in training cost *and* inference cost. If you save 3%, don't bother. Default tools: **SentencePiece BPE** (byte-level), vocab size 32K for small models, 128K+ for larger or more multilingual models. `tokenizers` from Hugging Face is the production-grade implementation. ## What training infrastructure do I need? Match to your scale: - **Single H100**: nanochat-scale only. Fine for <500M models, <50B tokens. - **8× H100 single node**: the solo-dev sweet spot. nanochat, TinyLlama, MicroLlama all train here. PyTorch FSDP2 is the right default. - **Multi-node**: when you outgrow 8× H100. DeepSpeed ZeRO-3, FSDP2 + tensor parallelism, or **torchtitan** (the recommended scale-out path in 2026 — native PyTorch, async checkpointing, scales to thousands of GPUs). - **Megatron-LM**: NVIDIA's reference for huge scale; rigid. Only use at >100 GPUs. Where to rent: Lambda Labs, RunPod, Vast.ai for spot 8× H100 at ~$15–20/hr. Modal and Beam are good for "I'd rather not manage infra at all" workflows but cost more. AWS / GCP / Azure are the most expensive on per-GPU-hour but easiest to fold into existing enterprise procurement. ## How does the post-training step work? Pre-training produces a *base model* — it completes text but doesn't follow instructions. To get a usable chat / assistant model, you post-train in three stages: **1. Supervised fine-tuning (SFT).** 10K–50K instruction examples in ChatML format. This teaches the model to respond to instructions rather than continue text. A single epoch is often enough. **2. Preference tuning.** 5K–20K preference pairs (good vs bad response to the same prompt). Use DPO, ORPO, KTO, or SimPO. ORPO merges SFT and DPO into one stage and is the simplest pipeline. The preference data is the bottleneck — generating it is harder than the training. **3. (Optional) Reinforcement learning with verifiable rewards.** GRPO or DAPO. Only for reasoning tasks where you can mechanically check the answer (math, code that compiles and passes tests). This is what powered DeepSeek R1's reasoning step-change. For most small-model projects, skip — the dataset cost is much higher than the gain on non-reasoning tasks. For a solo dev: SFT + ORPO is the right floor. Detailed framework coverage is in our [fine-tuning guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). ## Walk me through a $50 pre-training run This is roughly what happens when you run `speedrun.sh` on nanochat: 1. **Provision** an 8× H100 spot instance on Lambda or RunPod (~$15/hr). 2. **Train a tokenizer** on a slice of FineWeb (~10 minutes). 3. **Pre-train** a ~700M-parameter model on 40B FineWeb tokens (~1 hour). 4. **Mid-train** on a curated higher-quality mix (~15 minutes). 5. **SFT** on instruction data (~10 minutes). 6. **Optional RL** on GSM8K math (~15 minutes). 7. **Eval** against a standard benchmark suite (~5 minutes). 8. **Serve** via a web chat UI nanochat ships (~immediate). Total wall clock: ~3 hours. Total cost: ~$48 in compute (you'll also burn ~$5 on storage and ingress/egress). The output is a GPT-2-class model — not useful in production, but you understand the entire stack and you have a complete deployable artifact. **This is the right introduction to pre-training in 2026.** ## When does the math actually work out? The decision tree for "should I pre-train": 1. Do you have >10B clean domain-specific tokens that no open model has seen? *If no, fine-tune.* 2. Do you have $20K+ in compute budget for the training run? *If no, fine-tune.* 3. Have you exhausted fine-tuning and RAG on your problem? *If no, go back and try harder.* 4. Is the inference cost or sovereignty constraint dominant? *If yes, proceed.* 5. If you're doing this to learn — run nanochat for $50. Don't conflate "learning project" with "production project." The most common mistake is treating pre-training as a quality lever — "if I train my own, it'll be better than Llama." It will not be. The best open-weight models cost $50M+ to train and were built by teams of 30+ researchers over a year. Your $30K pre-training run will produce a model worse than the freely-available Llama 4 Scout, unless you have data Meta didn't. ## FAQ ### What's the cheapest way to actually pre-train a model end-to-end? ~$50 on an 8× H100 spot instance, using [karpathy/nanochat](https://github.com/karpathy/nanochat)'s `speedrun.sh`. You get a deployable GPT-2-quality model and complete understanding of the stack in about three hours. ### What's the difference between pre-training and fine-tuning? Pre-training builds the model from random initialization on huge web-scale data. Fine-tuning starts from someone else's pre-trained weights and adjusts them for a narrow task. Pre-training is the $10K–$1M operation; fine-tuning is the $1–$50 operation. For 99% of teams, fine-tuning is the right answer. ### How many tokens do I need to pre-train per billion parameters? Chinchilla compute-optimal is ~20:1 tokens per parameter. The 2026 inference-optimal practice for production small models is 100:1 to 1000:1 — Llama 3 used ~200:1, Liquid LFM2.5-350M used 80,000:1. For learning projects, 20:1 is enough. For production small models, push way past. ### What datasets should I use for pre-training? FineWeb (general web), FineWeb-Edu (educational filter for knowledge density), SlimPajama (cleaned, smaller scale), The Stack v2 (code), PG-19 (long-form prose). Mix domain-specifically. Deduplicate with `datatrove`. For tiny models, SmolLM-Corpus is pre-cleaned and ready. ### What's the modern small-LLM architecture I should copy? Decoder-only transformer + RMSNorm pre-norm + RoPE positional embeddings + SwiGLU FFN + grouped-query attention + no biases + tied embeddings for small models. Every 2026 open-weight model converged here. Don't innovate on architecture — innovate on data and training dynamics. ### Should I train my own tokenizer? Only if your domain compression rate is >15% better than the Llama tokenizer on your data. Otherwise adopt Llama 3, Qwen, or Mistral's tokenizer and inherit the proven vocabulary. ### What hardware do I need? Single H100 for tiny experiments. 8× H100 single node for serious solo work — this is the sweet spot. Multi-node only above that. Mac Studio Ultra is great for inference but ~10× slower than H100 for training; don't pre-train on a Mac unless it's a learning exercise. See our [Apple Silicon LLMs guide](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) for the inference side. ### What does it cost to train a Llama-3-class model? For 7B parameters at 15T tokens (Llama-3-8B scale): ~3M H100-hours, ~$1M–2M at spot pricing. Out of reach for solo devs without funding. For Chinchilla-optimal 7B at 140B tokens: ~$30K–50K at spot. For 1B at 50B tokens: ~$1.5K. For "GPT-2 quality" at 700M / 40B tokens: ~$50. ### Why bother pre-training at all if I can just fine-tune? You shouldn't, unless you have genuinely novel domain data, a data-sovereignty constraint, an edge-deployment niche that justifies a custom tiny model, or you're learning. For everything else, [fine-tune an existing open-weight model](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). ### How long does pre-training take? nanochat ~3 hours on 8× H100. TinyLlama 1.1B (3T tokens) ~90 days on 16× A100 historically; closer to 30–45 days on modern 8× H100. Llama-3-8B-class (15T tokens) ~6+ months on a multi-node cluster. The runtime is roughly linear in tokens-per-parameter times parameter-count. ### What's the right post-training stack in 2026? SFT (10K–50K instruction examples, ChatML) → ORPO or DPO (5K–20K preference pairs) → optional GRPO/DAPO for verifiable-reward reasoning tasks. ORPO merges SFT and DPO into one stage, simplest pipeline. The framework details (TRL, Axolotl, Unsloth, MLX-LM) are covered in our [fine-tuning guide](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). ## Related guides - [Fine-tuning LLMs](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/) — the practical alternative when pre-training is overkill - [Apple Silicon LLMs](https://codersera.com/blog/apple-silicon-llms-complete-guide-2026/) — running and fine-tuning models on Mac - [Self-hosting LLMs](https://codersera.com/blog/self-hosting-llms-complete-guide-2026/) — serving infrastructure once you have a model - [Open-source LLMs landscape](https://codersera.com/blog/open-source-llms-landscape-2026/) — the open-weight family you'd otherwise fine-tune - [Llama 4 guide](https://codersera.com/blog/llama-4-complete-guide-2026/) - [DeepSeek V4 guide](https://codersera.com/blog/deepseek-v4-complete-guide-2026/) - [Qwen 3.5 guide](https://codersera.com/blog/qwen-3-5-complete-guide-2026/) =============================================================== [17/20] Android Emulators in 2026: The Complete Guide for Developers URL: https://codersera.com/blog/android-emulators-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== *Last updated: May 1, 2026* The Android emulator landscape in 2026 is wider and weirder than most people realise. A QA engineer at a fintech, a casual gamer trying to run a gacha title, an indie React Native developer on an M3 MacBook Air, and a Linux power user on Fedora 41 will all reach for very different tools — and most of the "best Android emulator" lists online happily ignore that those use cases barely overlap. This guide is the version we wished existed: a pragmatic, opinionated tour of every emulator class that actually matters in 2026, with real hardware requirements, real pricing, and real trade-offs. ## TL;DR - **Building or testing an Android app:** Android Studio AVD locally for daily work; Firebase Test Lab, AWS Device Farm, BrowserStack, or Sauce Labs for matrix coverage on real devices. - **Playing Android games on a PC:** LDPlayer 9 or BlueStacks 5 on Windows; MuMu Player on a Mac; GameLoop only for Tencent titles that demand it. - **Apple Silicon Mac (M1–M4):** Android Studio's native ARM64 emulator is excellent. BlueStacks Air also runs natively. Avoid Intel-only x86 emulators that need Rosetta. - **Linux desktop:** Waydroid (container-based, near-native) for personal use; Android Studio AVD for development. - **Low-end PC, no virtualization:** See our deep dive on [Android emulators without VT](https://codersera.com/blog/10-best-android-emulators-for-pc-without-virtualization-technology-vt/) and the [best emulator for low-end PCs in 2026](https://codersera.com/blog/best-android-emulator-for-low-end-pc-2026/). - **No install at all:** Browser-based and cloud emulators — see the [browser-free emulators guide](https://codersera.com/blog/android-emulator-online-browser-free/) and the [cloud phone emulators deep dive](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). ## What an Android emulator actually is (and isn't) An Android emulator runs an Android OS image inside a virtual machine on your computer. Strictly, the term "emulator" is a small misnomer — Google's AVD, Genymotion, BlueStacks, MEmu, LDPlayer and friends all use hardware virtualization (Intel VT-x, AMD-V, or Apple's Hypervisor.framework) to execute Android natively rather than interpreting it instruction-by-instruction. That is why a 2026-vintage emulator can hit 60 FPS in a 3D game on midrange hardware, and why a 10-year-old VirtualBox-based Android image cannot. The other thing emulators are not: they are not the same as *simulators*. iOS simulators (the kind Xcode ships) run an iOS-flavoured binary on your Mac's host CPU and rely on framework shims. Android emulators boot a real Android system image — kernel, system_server, ART, the lot — which is why bug repros from an emulator are usually trustworthy. ## Who actually uses Android emulators, and why their needs differ Lumping these audiences together is how you end up reading "BlueStacks is the best emulator" advice when you're trying to debug a memory leak in a React Native app. They are not the same job. - **App developers** need the Google Android Emulator (AVD), Play Store images, fast hot-reload, debuggable system images, and the ability to spin up arbitrary API levels (Android 6 through 16). Latency and accuracy matter; FPS does not. - **QA and automation engineers** need parallel device matrices, headless runs in CI, fingerprint diversity (manufacturer, locale, screen DPI), and access to real devices for the long tail of OEM bugs. They live in Firebase Test Lab, AWS Device Farm, BrowserStack, and Sauce Labs. - **Mobile gamers** need raw FPS, low input lag, mouse-and-keyboard mapping, multi-instance, and macro support. They reach for LDPlayer, BlueStacks, MEmu, MuMu, and GameLoop. - **Casual users** running messaging apps or banking apps on a desktop want a "just works" install, low resource footprint, and ideally no Google account weirdness. Browser-based and cloud emulators dominate this segment. - **Security researchers and reverse engineers** need rooted images, x86_64 builds with full debugging surface, and frequently use Genymotion or custom AOSP builds. If a recommendation doesn't name your role, it's probably wrong for you. ## The full taxonomy of Android emulators in 2026 ### Official: Android Studio AVD Google's Android Virtual Device manager, bundled with Android Studio, is the only emulator that ships system images blessed by the Android team. In 2026 it supports API 23 (Android 6) through API 36 (Android 16), with native ARM64 system images for Apple Silicon and Linux ARM hosts. It uses Intel HAXM (deprecated, EOL 2026), Windows Hypervisor Platform (WHPX), AEHD on Windows, KVM on Linux, and Hypervisor.framework on macOS. Free. ### Consumer gaming emulators BlueStacks (now in versions 5 and 10/X), LDPlayer 9, NoxPlayer, MEmu Play, GameLoop (Tencent), and MuMu Player 12 (NetEase) all target gamers. They diverge mostly on: which Android version they ship, how aggressively they monetise via in-app ads, how clean their installer is, and how well their key-mapping editor works. ### Developer-grade alternatives Genymotion Desktop and Genymotion SaaS sit between AVD and the gaming emulators — fast to boot, scriptable via Java/Python APIs, with sensor injection (GPS, battery, network throttling). Used heavily for automated UI testing pre-Firebase Test Lab. ### Linux container approach Waydroid runs Android 13 inside an LXC container on a Wayland Linux host, with direct hardware access via the binder interface. No virtualization overhead — the Android userland is just another set of Linux processes. Free and open source. ### Browser and cloud emulators ApkOnline, appetize.io, and various cloud phone services stream a remote Android instance to a browser. No install, no VT requirement, but you are sending input and receiving video over the internet. Covered in detail in our [ApkOnline guide](https://codersera.com/blog/apkonline-android-emulator-a-comprehensive-guide/), the [free browser emulators guide](https://codersera.com/blog/android-emulator-online-browser-free/), and the [cloud phone emulators deep dive](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). For the broader category — Chrome-based options included — see [Android emulators for Chrome](https://codersera.com/blog/best-android-emulators-for-chrome-run-mobile-apps-in-your-browser/). ### Cross-OS emulators (iOS-on-Android, Android-on-iOS) The "iPhone emulator for Windows" and "iOS emulator for Mac" categories are mostly streaming services or Xcode itself. Real iOS emulation on non-Apple hardware is largely a myth — see our explainers on the [best iPhone emulator for Windows PC](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/) and the [best iOS emulator for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/). For a side-by-side of the wider mobile-emulator universe, the [32 mobile emulators review](https://codersera.com/blog/32-best-mobile-phone-emulators-reviewed-in-2025-android-ios/) is the long index. ## Consumer emulator comparison | Emulator | Vendor | Latest Android version | Min RAM | Min disk | VT required | Mac (Apple Silicon) | Price | | --- | --- | --- | --- | --- | --- | --- | --- | | BlueStacks 5 | now.gg | Android 11 (Pie 32-bit also) | 4 GB | 5 GB | Recommended | BlueStacks Air (native) | Free; Premium ad-free $4/mo | | BlueStacks X / 10 | now.gg | Cloud-streamed | 2 GB (thin client) | n/a | No | Browser | Free with ads | | LDPlayer 9 | XUANZHI | Android 9 / 11 | 2 GB (4 GB rec.) | 36 GB recommended | Yes | No | Free (ad-supported) | | NoxPlayer | Bignox | Android 9 | 2 GB | 3 GB | Recommended | Intel Macs only | Free | | MEmu Play | Microvirt | Android 9 (multi-kernel) | 2 GB | 2 GB | Recommended | No | Free | | GameLoop | Tencent | Android 7 | 3 GB | 1.5 GB | Yes | No | Free | | MuMu Player 12 | NetEase | Android 12 (x86_64) | 4 GB | 5 GB | Yes (i5-7500+) | MuMu Pro for Mac (native ARM) | Free | Two things worth calling out. First, the "free" label hides a wide range of monetisation. BlueStacks runs in-emulator ads in the launcher unless you pay for Premium; LDPlayer's installer recommends partner apps; GameLoop is mostly clean but only because Tencent makes its money once you're inside Honor of Kings or PUBG Mobile. Second, BlueStacks 10 / X is not "newer" than BlueStacks 5 in the way a normal version number suggests — it is a separate cloud-streaming product, and BlueStacks 5 remains the locally-installed flagship. ## Developer and QA emulator comparison | Tool | Type | Best for | Pricing (2026) | Real devices | | --- | --- | --- | --- | --- | | Android Studio AVD | Local emulator | Daily dev, debugger attach, hot reload | Free | No | | Genymotion Desktop | Local emulator (VirtualBox) | Sensor injection, sandboxed test runs | Free for personal; Indie €136/yr; Business €412/yr | No | | Genymotion SaaS | Cloud emulator | CI matrix, scripted teardown | $0.06/minute pay-as-you-go | No | | Firebase Test Lab | Google device farm | Robo tests, Espresso, low-cost CI | Free 10 virtual / 5 physical per day; $1/hr virtual, $5/hr physical | Yes | | AWS Device Farm | AWS device farm | Heavy parallel test runs | $0.17/device-minute or $250/month per slot (unmetered) | Yes | | BrowserStack App Live / Automate | Real device cloud | Manual + automated, large device library | Live from $29/mo; Automate from $129/mo per parallel | 2,500+ Android | | Sauce Labs | Emulator + real device cloud | Enterprise scale, secure tunnels | Virtual Cloud $149/mo; Real Device $199/mo | 7,500+ devices | The right answer for most teams is: **AVD locally + Firebase Test Lab in CI** until the Firebase device variety stops being enough, then add BrowserStack or Sauce Labs. AWS Device Farm is competitive when your stack is already on AWS or when unmetered slots beat per-minute economics. ## Hardware: what your machine actually needs The dominant variable in 2026 is whether your CPU exposes Second Level Address Translation (SLAT) and whether your OS lets the emulator use it. Anything Intel Core i3 (Haswell, 2013+) or AMD Ryzen will have SLAT; the trouble is usually that VT-x or AMD-V is disabled in the BIOS, or that Windows 11 24H2's expanded Virtualization-Based Security has claimed the hypervisor and starved your emulator of it. One important deprecation: **Google's Android Emulator Hypervisor Driver (AEHD) is being sunset on December 31, 2026**. Windows users on AVD need to migrate to the Windows Hypervisor Platform before then. Microsoft also killed Windows Subsystem for Android in March 2025, so WSA is no longer an option for running Android apps on Windows. | Use case | Min CPU | Min RAM | Recommended RAM | Disk | Notes | | --- | --- | --- | --- | --- | --- | | Single AVD (Android Studio) | i3 8th gen / Ryzen 3 / Apple M1 | 8 GB | 16 GB | ~6 GB per AVD + 8 GB SDK | SSD strongly recommended | | Multi-instance gaming (BlueStacks/LDPlayer) | i5 / Ryzen 5 | 8 GB | 16 GB | 10–40 GB | Dedicated GPU helps but not required | | Heavy QA matrix locally | i7 / Ryzen 7 / M2 Pro | 16 GB | 32 GB | 50+ GB | NVMe SSD essential | | Low-end PC / no VT | any 64-bit dual-core | 4 GB | 8 GB | 5 GB | Cloud or no-VT options only | | Apple Silicon (M1–M4) | any M-series | 8 GB | 16 GB | 15 GB | Use ARM64 system images | ## Mac vs Windows vs Linux vs Chromebook **Windows** is still the broadest target. Every consumer emulator ships a Windows installer first; AVD, Genymotion, BlueStacks, LDPlayer, MEmu, NoxPlayer, MuMu, and GameLoop all run there. The catch is the Hyper-V tug-of-war: WSL2, Docker Desktop, Windows Sandbox, and any "Memory Integrity" toggle all reserve the hypervisor, and many gaming emulators refuse to launch when Hyper-V is on. BlueStacks specifically ships a Hyper-V build to work around this. **macOS on Apple Silicon** is, somewhat surprisingly, an excellent Android emulation target — but only if the emulator ships a native ARM64 build. Android Studio AVD, MuMu Pro for Mac, BlueStacks Air, and Genymotion all qualify. NoxPlayer, MEmu, LDPlayer, and GameLoop do not have native Apple Silicon builds and either run only on Intel Macs or rely on x86 translation that punishes performance. Allocate at least 4 cores and 6–8 GB of RAM to the VM for smooth UI animations. **Linux** is the home territory of two very different choices. Android Studio AVD on KVM is fast, scriptable, and the standard for development. Waydroid is a different model entirely: it boots Android 13 inside an LXC container, sharing the host kernel, with effectively zero virtualization overhead. Waydroid is a great daily-driver for casual Android-app use on a Linux desktop and works on x86_64 and ARM64 hosts. **ChromeOS** already runs Android apps natively via the bundled Play Store runtime (ARC), which is great for users but offers no developer surface. To do real development on a Chromebook, enable Linux (Crostini) and run Android Studio AVD or Waydroid inside it; or use a cloud emulator service. See [Android emulators for Chrome](https://codersera.com/blog/best-android-emulators-for-chrome-run-mobile-apps-in-your-browser/) for the browser-only routes. ## Security and privacy: the parts vendors don't volunteer Three honest observations. **Antivirus false positives are constant.** Emulators install kernel drivers, modify hypervisor state, and inject input — behaviour that looks identical to malware to most heuristic engines. Defender flagging BlueStacks or LDPlayer is not, by itself, a sign of compromise. The actual rule: only download from the vendor's official domain, and verify the installer signature. **Bundled software is the real risk.** Several consumer emulators have shipped third-party adware or partner-app installers in the past. NoxPlayer notably had a 2021 supply-chain compromise (the BigNox update server was breached and used to ship a malicious update). BlueStacks, LDPlayer, and MEmu have all shipped opt-out installers for partner products at various times. Read the installer screens; uncheck the recommended apps. **Don't sign in with your real Google account.** Most consumer emulators require a Google account to use the Play Store. Use a throwaway. Don't connect financial apps to a desktop emulator for any reason — many banking apps refuse to run on emulators, and the ones that don't refuse do not consider the desktop a trusted device. Developer-grade tools (AVD, Genymotion, Firebase Test Lab, AWS Device Farm) have a substantially cleaner posture: no in-emulator ads, no bundled partner apps, signed installers, and clear data-handling agreements. ## The mobile QA testing emulator stack If you ship an Android app, the question isn't "which emulator" — it's "which combination of emulator and real-device cloud." 1. **Local development:** Android Studio AVD, one or two API levels (current and current-minus-2), Play Store image when you need Google services. 2. **Pre-merge CI:** Firebase Test Lab Robo crawler against virtual devices ($1/hr) catches the obvious crashes cheaply. Espresso + UI Automator suites run here too. 3. **Release candidate matrix:** Firebase Test Lab physical devices ($5/hr) or AWS Device Farm slots for the OEM-specific bugs (Samsung One UI, Xiaomi MIUI / HyperOS, Oppo ColorOS) that virtual devices cannot reproduce. 4. **Manual exploratory:** BrowserStack App Live or Sauce Labs Real Device Cloud to drive a specific Galaxy S24 / Pixel 9 / Redmi Note 13 by hand when a customer report mentions one. 5. **Performance and battery:** Real devices, full stop. Emulators do not faithfully model thermal throttling, modem behaviour, or battery curves. ## Known issues and what to avoid - **Windows 11 24H2 + emulator crashes.** Memory Integrity / VBS reserves the hypervisor; LDPlayer and MEmu often fail to launch. Either disable Memory Integrity (security trade-off) or move to a Hyper-V-aware emulator (BlueStacks, AVD with WHPX). - **Intel HAXM is gone.** If a tutorial tells you to install HAXM, the tutorial is out of date. Use WHPX on Windows, KVM on Linux, Hypervisor.framework on macOS. - **NoxPlayer supply chain history.** The BigNox 2021 incident is a cautionary tale; if you must use Nox, verify hashes and keep it isolated from work machines. - **x86 emulators on Apple Silicon.** Performance is poor and several emulators flat-out crash. Use ARM-native builds. - **"Free iOS emulator for PC" downloads.** Almost universally junkware. Real iOS development happens on a Mac with Xcode; everything else is streaming or fake. - **WSA is gone.** Windows Subsystem for Android was retired by Microsoft on March 5, 2025. Don't plan around it. - **Banking and DRM apps.** SafetyNet and Play Integrity API will flag emulators. Don't waste days trying to bypass — it's a moving target and against most apps' ToS. ## FAQ ### Is Android Studio's emulator good enough for daily development? Yes. With native ARM64 system images on Apple Silicon and WHPX/KVM acceleration on Windows and Linux, AVD boots in under 15 seconds on modern hardware and supports every API level you'll need. The friction is mostly memory: 16 GB host RAM is the comfortable floor. ### Which Android emulator is best for low-end PCs? LDPlayer 9 Lite, MEmu Lite, and NoxPlayer's lightweight profile run on 4 GB RAM machines with VT enabled. Without VT, you're looking at cloud or browser-based options. Our [low-end PC emulator guide](https://codersera.com/blog/best-android-emulator-for-low-end-pc-2026/) goes deeper. ### Can I run an Android emulator without VT-x / virtualization? Locally, performance will be unusable for most apps; the few options that work are limited. Cloud emulators bypass the requirement entirely. See [our list of no-VT emulators](https://codersera.com/blog/10-best-android-emulators-for-pc-without-virtualization-technology-vt/). ### What's the best Android emulator for a Mac with Apple Silicon? For development, Android Studio AVD with an ARM64 system image. For gaming, BlueStacks Air (native Apple Silicon build) or MuMu Pro for Mac. Avoid NoxPlayer, LDPlayer, and MEmu — they have no native ARM build. ### Is BlueStacks safe? The BlueStacks installer from bluestacks.com is signed, malware-free, and used by tens of millions of people. The risks are: antivirus false positives, in-emulator ads, and whatever Android apps you install inside it. Don't use it for banking, and use a throwaway Google account. ### Is NoxPlayer safe in 2026? NoxPlayer's 2021 supply-chain breach (NightScout) means it sits lower on the trust ladder than its competitors. If you use it, isolate it, never sign in with a real Google account, and prefer alternatives for sensitive workloads. ### What happened to Windows Subsystem for Android? Microsoft retired WSA on March 5, 2025. App support ended; there is no successor on Windows 11. For Android-on-Windows now, use BlueStacks, LDPlayer, or AVD. ### How much does Firebase Test Lab cost? Free tier: 10 virtual-device tests and 5 physical-device tests per day. Beyond that: $1 per virtual-device hour, $5 per physical-device hour, billed per minute. ### How much does AWS Device Farm cost? $0.17 per device-minute pay-as-you-go, or $250 per month per device slot for unmetered. The first 1,000 device-minutes are free. ### BrowserStack vs Sauce Labs — which should we pick? BrowserStack publishes lower entry pricing (App Live from $29/mo, Automate from $129/mo per parallel) and has the larger Android real-device library. Sauce Labs is more competitive at enterprise scale (100+ parallels) and includes 1,700+ emulators alongside its real-device cloud. For most teams under 20 parallels, BrowserStack wins on price transparency. ### Can I use Genymotion for free? Genymotion Desktop has a free tier for personal use. Commercial use requires Indie (€136/yr), Business (€412/yr), or Enterprise. Genymotion SaaS is pay-as-you-go at $0.06 per device-minute. ### Is Waydroid a real alternative to AVD on Linux? For running Android apps on a Linux desktop — yes. For Android development with a debugger — no, AVD remains the right tool. Waydroid is great when you want one or two Android apps living alongside your Linux apps with near-native performance. ### Can I emulate iOS on Windows or Android? Not really. Tools that claim to are usually skinned Android emulators with iOS-themed launchers, or they are remote streaming services. Real iOS testing requires Xcode on a Mac, or a service like BrowserStack streaming a real iPhone. See [iPhone emulators for Windows](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/) and [iOS emulators for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/). ### What about Chromebooks? Chromebooks already run Android apps natively via ARC. For development, enable Crostini Linux and run Android Studio AVD inside it, or use a cloud emulator. See [Android emulators for Chrome](https://codersera.com/blog/best-android-emulators-for-chrome-run-mobile-apps-in-your-browser/). ### Are browser-based emulators good enough for testing? For sanity-checking a build or a quick demo, yes. For automated testing against real OS behaviours, no — latency, audio, and sensor support are all limited. Background reading: [free browser emulators](https://codersera.com/blog/android-emulator-online-browser-free/), [ApkOnline guide](https://codersera.com/blog/apkonline-android-emulator-a-comprehensive-guide/), and [cloud phone emulators](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). ### What's the difference between an emulator and a simulator? An emulator boots a real Android OS image inside a VM and runs unmodified Android binaries. A simulator (the iOS kind) runs a re-compiled-for-host version of the OS and uses framework shims. Android emulators are closer to "real" devices than iOS simulators are, which is why Android engineers trust emulator bug repros more than iOS engineers trust simulator ones. ## Next steps If you're building or testing an Android app and want senior people who already know the AVD-vs-Firebase-vs-BrowserStack trade-offs cold — and who can ship without three weeks of ramp-up — we can help. [Hire a Codersera-vetted React Native or Android developer](https://codersera.com/blog/hire/hire-react-native-developer). We pre-vet for technical fit, remote-readiness, and the kind of platform judgement this guide is full of, so you spend less time interviewing and more time shipping. For the wider mobile-emulator universe, our [review of 32 mobile phone emulators](https://codersera.com/blog/32-best-mobile-phone-emulators-reviewed-in-2025-android-ios/) is the best long-form index to keep bookmarked alongside this guide. =============================================================== [18/20] iOS Simulators in 2026: The Complete Guide for Developers URL: https://codersera.com/blog/ios-simulators-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== If you build for the App Store, you live inside the iOS Simulator. It is the single most-used tool in the Apple developer toolchain, and it is also one of the most misunderstood. People reach for an "iOS emulator for Windows" the way they reach for an Android emulator, and almost everything they find is either an Android skin, a streaming service, or outright malware. There is exactly one place a real iOS Simulator runs: macOS. Everything else is a workaround. This guide covers what the Xcode Simulator is, your options on Windows and Linux (and which ones are scams), the legitimate cloud services that fill the gap, and the day-to-day Simulator features, automation, and CI patterns that real iOS teams use in 2026. *Last updated: May 1, 2026.* ## TL;DR - The Xcode Simulator is a **simulator**, not an emulator. It runs your app as a native macOS process against iOS frameworks. It does not virtualise iPhone hardware, so it is fast, and it does not boot a real iOS image, so it cannot run device-only binaries. - There is no real iOS Simulator on Windows or Linux. Anything advertised as one is either an online streaming wrapper around a real Mac, an Android launcher reskinned to look like iOS, or unsafe. - Legitimate non-Mac options fall into two buckets: **browser-streamed simulators** (Appetize.io, Smartface) and **real-device clouds** (BrowserStack App Live, Sauce Labs, AWS Device Farm, Firebase Test Lab, Kobiton, LambdaTest, Corellium). - For real teams, the workflow that scales is: develop against the local Simulator, automate with `xcrun simctl` and fastlane, run CI on macOS GitHub Actions runners with Xcode 26, and reach for a real-device cloud for release-blocking smoke tests. - If you need actual iOS engineering capacity, [hire a Codersera-vetted iOS or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer) rather than fighting the toolchain alone. ## 1. What the iOS Simulator Actually Is When you boot an iPhone in Xcode, you are not running a virtual machine. The Simulator is a regular macOS application called `Simulator.app` that loads a build of iOS frameworks compiled for the host CPU. Your app, when built for the Simulator, is also compiled for the host CPU (x86_64 on Intel Macs, arm64 on Apple Silicon). The two link together and run as a normal Mach-O process. That is why iOS builds have an `iphonesimulator` SDK distinct from `iphoneos`: the *same source code* produces different binaries depending on which CPU you target. There is no instruction translation, no hypervisor, no Rosetta-style layer. It is fast precisely because it isn't pretending to be iPhone hardware. This has consequences: - **Performance is unrealistic.** The Simulator inherits your Mac's CPU, GPU, RAM, and SSD. A scrolling animation that runs at 120fps on an M4 may stutter on an iPhone 12. - **Hardware features are simulated, not present.** No real camera frames, no real GPS, no real cellular radio, no real Secure Enclave, no real Bluetooth peripherals. - **Some libraries don't exist.** Pre-compiled vendor SDKs that ship only device slices won't link; that is why most XCFrameworks publish both slices. - **Some bugs only happen on device.** Memory pressure, thermal throttling, code-signing entitlements, push tokens, and StoreKit live transactions are device-only territory. The Simulator is for the 95% of UI, layout, navigation, and logic work where the host's speed is a feature. ## 2. Simulator vs. Emulator: Why Android Does It Differently The Android Emulator runs an actual ARM/x86 Android image inside QEMU on top of Hypervisor.framework, HAXM/WHPX, or KVM. It boots a real Android kernel. Your APK is the same APK that runs on a Pixel — full-fat *emulation* with hardware virtualisation: high fidelity, heavy on the host. The iOS Simulator does the opposite trade. It throws away fidelity for speed. You build for `iphonesimulator` and a macOS process appears — no kernel, no boot, no virtualisation. If you want a side-by-side primer on the Android side, see our [Android emulators complete guide for 2026](https://codersera.com/blog/android-emulators-complete-guide-2026/). The short version: an Android emulator is a virtual phone; the iOS Simulator is your app pretending to be on a phone. ## 3. Installing and Using the Xcode Simulator The only supported way to get the iOS Simulator is to install Xcode on macOS. As of May 2026, Xcode 26.4.1 is the current release, free from the Mac App Store or the Apple Developer downloads page. It bundles the Simulator runtime for the latest iOS, plus an installer for older iOS, watchOS, tvOS, and visionOS runtimes. You can manage runtimes from *Xcode → Settings → Components*. Day-to-day Simulator features that matter: - **Device chooser.** Pick any iPhone or iPad form factor on any installed iOS runtime. Xcode 26 ships device profiles for the iPhone 17 lineup and the iPad Pro M5. - **Network Link Conditioner.** Bundled in the *Additional Tools for Xcode* download. Throttle bandwidth and inject packet loss with presets for 3G, Edge, LTE, 5G, and "Very Bad Network." It applies to the host, so it affects every Simulator. - **Location simulation.** *Features → Location* lets you set a static GPS coordinate, simulate a freeway drive, a city run, or import a GPX file for arbitrary tracks. Critical for any maps or fitness app. - **Push notification simulator.** Drag an APNs JSON payload onto the Simulator window, or run `xcrun simctl push <device> <bundle-id> payload.json`. No APNs server, no certificates required for local testing. - **Device states.** Toggle Light/Dark mode, Dynamic Type sizes, accessibility settings, low-power mode (cosmetic), and the new Liquid Glass appearance toggle Apple introduced in iOS 26. - **Scenarios.** Simulate incoming calls, memory warnings, screenshot triggering, and Handoff between devices. - **Hardware menu.** Rotate, shake, simulate Touch ID and Face ID, trigger Home and Lock buttons, and toggle Increase Contrast. - **Drag-and-drop media.** Drop photos, videos, contacts (.vcf), or app payloads on the Simulator to install them. - **Recording.** *File → Record Screen* for video, ⌘S for screenshots, both wired into `simctl io` for headless capture. ## 4. Automating the Simulator with `xcrun simctl` `simctl` is the command-line interface to the Simulator. It is the foundation under fastlane, Detox, Maestro, and most CI scripts. Useful commands: ``` # List everything xcrun simctl list devices available # Create a fresh device xcrun simctl create "QA-iPhone-17" "iPhone 17 Pro" "com.apple.CoreSimulator.SimRuntime.iOS-26-0" # Boot, install, launch xcrun simctl boot "QA-iPhone-17" xcrun simctl install booted ./build/MyApp.app xcrun simctl launch booted com.codersera.myapp # Send a push payload xcrun simctl push booted com.codersera.myapp payload.json # Set location xcrun simctl location booted set 37.3349,-122.0090 # Open a deep link xcrun simctl openurl booted "myapp://orders/42" # Capture xcrun simctl io booted screenshot out.png xcrun simctl io booted recordVideo --codec=h264 out.mp4 # Reset xcrun simctl shutdown all xcrun simctl erase all ``` Anything you can do in the Simulator UI, `simctl` can script. CI pipelines should treat Simulators as ephemeral: create, boot, run tests, shut down, erase. Never share a Simulator between test runs — stale UserDefaults and keychain state are the most common source of flaky iOS tests. ## 5. *fastlane snapshot* and Screenshot Automation fastlane `snapshot` (the modern action is `capture_screenshots`) drives a UI test target across every device and locale you list, producing the exact screenshots the App Store expects. For 2026 submissions, you need **1320×2868** for the iPhone 6.9-inch slot and **2064×2752** for the 13-inch iPad. fastlane 2.230.0 added support for the iOS 26 device sizes and the `TabView` identifier changes Apple shipped at WWDC 25; older fastlane versions silently fail on iOS 26 simulators. The pattern looks like: ``` # fastlane/Snapfile devices(["iPhone 17 Pro Max", "iPhone 17", "iPad Pro 13-inch (M5)"]) languages(["en-US", "es-ES", "ja-JP", "de-DE"]) scheme("MyAppUITests") output_directory("./screenshots") clear_previous_screenshots(true) ``` Combined with `deliver`, you push localised screenshots straight into App Store Connect. For teams shipping multi-locale apps, this is the difference between a one-hour release ritual and a one-week marketing-team bottleneck. ## 6. CI: GitHub Actions, Xcode Cloud, Bitrise iOS CI requires a macOS runner. There are no Linux shortcuts; `xcodebuild` is macOS-only. The three real options: - **GitHub Actions `macos-15` / `macos-26` runners.** Apple Silicon, Xcode 26 pre-installed. Apple Silicon GitHub runners cut iOS test wall time roughly in half compared to the older Intel `macos-12` images. Split builds with `xcodebuild build-for-testing` followed by `test-without-building` to cache the build artefact across parallel test shards. - **Xcode Cloud.** Apple's first-party CI, billed in compute hours and tied to App Store Connect. Lowest friction for pure Apple shops; least flexible for monorepos and hybrid stacks. - **Bitrise / Codemagic / CircleCI macOS.** Specialised mobile CI with dedicated Mac infrastructure. Better caching primitives than generic CI for Pods, SPM, and DerivedData; more expensive per minute. A representative GitHub Actions step: ``` jobs: test: runs-on: macos-26 steps: - uses: actions/checkout@v4 - uses: maxim-lobanov/setup-xcode@v1 with: xcode-version: "26.4" - name: Build for testing run: | xcodebuild build-for-testing \ -scheme MyApp \ -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.0' - name: Test run: | xcodebuild test-without-building \ -scheme MyApp \ -destination 'platform=iOS Simulator,name=iPhone 17,OS=26.0' \ -resultBundlePath TestResults.xcresult ``` ## 7. iOS on Windows and Linux: The Honest Answer You cannot run a real iOS Simulator on Windows or Linux. Apple does not ship one, and the iOS frameworks are not redistributable. What does exist falls into four categories, only the last two of which are useful for development: 1. **Android skins dressed as iOS.** Apps like the historical "iEMU" or various "iOS launcher" APKs. They run on Android and theme it. They cannot run iOS apps. Some are outright malware. Avoid. 2. **Web shells with screenshots of iOS.** "iPadian" is the canonical example: a Windows app that shows iOS-shaped UI built in Adobe AIR or .NET. It does not run iPhone apps; it runs a curated catalogue of look-alike web apps. Functionally entertainment, not development. 3. **Browser-streamed Simulators on real Macs.** Appetize.io, Smartface Cloud, BrowserStack App Live: a real Mac runs the real Simulator and streams the screen to your browser. This is legitimate, useful for demos and quick checks, and what most "iOS emulator online" reviews are actually pointing at, often without saying so. 4. **Real iOS device clouds.** Hosts of physical iPhones you connect to remotely. Sauce Labs, BrowserStack, AWS Device Farm, Firebase Test Lab, Kobiton, LambdaTest, Corellium. Slower than a local Simulator, far higher fidelity than any browser stream. For deeper context on what people pick when they want any of this on Windows specifically, see our [best iPhone emulator for Windows PC](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/) roundup and the [10 best free iPhone emulators online](https://codersera.com/blog/10-best-free-iphone-emulators-online-2025-no-download-required/) shortlist. On Mac specifically, our [best iOS emulator for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/) guide covers when reaching for anything other than the bundled Xcode Simulator makes sense. ## 8. Online iOS Emulators and Browser-Based Simulators "Online iOS emulator" almost always means: a browser interface streaming an iOS Simulator (or a real iPhone) running on someone else's hardware. The economics: someone has to own a Mac, you pay them by the minute. That is the entire business model. Pricing as of May 2026: | Service | What you get | Entry price | Honest fit | | --- | --- | --- | --- | | Appetize.io | Streamed Xcode Simulator, embeddable in a webpage, Playwright automation | Free tier (100 min/month, queued); Basic from $40/month; Premium $400/month; Enterprise from $2,000/month | Demos, sales engineering, embedded interactive tutorials, automated regression with Playwright | | Smartface Cloud | Cloud iOS dev IDE on Windows/Linux; runs real iOS builds remotely | From around $99/month per developer, enterprise quoted | JavaScript/TypeScript native iOS development from a non-Mac | | BrowserStack App Live | Real iPhones (not Simulators) streamed in browser | $29/user/month for App Live | Manual QA on real hardware, last-mile pre-release smoke testing | | LambdaTest (TestMu AI) Real Device Cloud | Real iOS devices in browser plus automation hooks | From $39/month entry tier | Cost-conscious teams that want both manual and automated mobile testing | | iPadian / iEMU / "free iOS emulators" | Not iOS. Skinned Windows or Android UIs. | Free / ad-supported | Avoid. See section 9. | For a longer survey including more niche entries, our [free iPhone emulators online comprehensive guide](https://codersera.com/blog/free-iphone-emulators-online-a-comprehensive-guide/) and [best virtual mobile device emulators](https://codersera.com/blog/best-virtual-mobile-device-emulators-run-android-and-ios-apps/) piece walk through specific UI quirks and trial limits. ## 9. Known Issues and What to Avoid (the iOS-Emulator-for-Windows Scams) Search "iOS emulator Windows" on any major engine and the first page is almost entirely SEO content optimised for the keyword, not for the developer. Some patterns to watch for: - **Names that recycle dead products.** iPadian, iEMU, AIR iPhone, Cider, MobiOne. None of these are functioning iOS environments today. Several of the modern download pages bearing these names serve installers signed with throwaway certificates. - **"Free download, no Mac required."** Real iOS Simulators require iOS framework binaries that Apple does not redistribute. A free, signature-light installer for an "iOS emulator" is almost always either an empty Electron / .NET shell or a malware dropper. - **Survey walls and "human verification."** A legitimate developer tool never asks you to fill in a CPA offer to unlock a download. - **YouTube tutorials with millions of views.** The economics of monetised YouTube reward "I ran iOS on Windows!" thumbnails. Most of these videos demonstrate iPadian (a web wrapper) or an Android emulator booted into a custom launcher. - **"Cloud iOS" with a flat $5 lifetime fee.** Mac Mini hosting costs more than that per month wholesale. The math doesn't work; the service either disappears in weeks or it's running on stolen credentials. Three rules that filter 95% of the noise: 1. If it doesn't require a real Apple Developer Program account or device serials, it isn't running real iOS. 2. If it claims to run arbitrary `.ipa` files in your browser without uploading them to a real iOS host, it's lying. 3. If the company has no LinkedIn footprint, no funding history, and no enterprise customers, treat the binary as untrusted. ## 10. Real-Device Clouds: When Simulators Aren't Enough Simulators cannot certify a release. Camera capture, ARKit, CoreNFC, real cellular conditions, real APNs delivery, real StoreKit purchases, thermal throttling under load — all device-only. Real-device clouds rent you a physical iPhone over the network. | Provider | Pricing model | Indicative price | Strengths | Watch-outs | | --- | --- | --- | --- | --- | | BrowserStack App Live / App Automate | Per-user or per-parallel | App Live $29/user/month; App Automate Pro $249/month | Largest device fleet (30,000+), instant access, strong manual QA UX | Per-parallel pricing scales fast for automated suites | | Sauce Labs Real Device Cloud | Concurrency + minutes | Typically $3,000–$75,000+ /year, quote-based | Enterprise-grade, private device options, mature Appium pipelines | No published self-serve tier; sales-led | | AWS Device Farm | Pay-as-you-go or unmetered slot | $0.17/device-minute or $250/slot/month | Pure usage billing, integrates with AWS IAM, no minimums | Smaller iOS fleet; older devices over-represented | | Firebase Test Lab (Apple devices) | Per-hour | $5/hr real device, $1/hr virtual; free tier 5 real / 10 virtual tests/day | Cheapest entry, deep Google Cloud integration, good for XCTest | Limited device list, no manual exploratory mode | | Kobiton | Per-user / per-device subscription | Quote-based, mid-market positioning | Visual testing add-ons, on-prem option for regulated industries | Higher per-device cost than AWS or Firebase | | LambdaTest (TestMu AI) | Tiered subscription | From $39/month entry; HyperExecute and AI test agents at higher tiers | Cheapest premium-feeling self-serve, broad cross-browser overlap | Newer iOS device additions sometimes lag | | Corellium | Virtualised iOS on ARM hosts | Quote-based, security-research positioning | Snapshot, fuzz, jailbreak research, kernel debugging | Not for App Store-flow testing; no App Store / TestFlight on virtual devices | For most product teams the right shape is: develop on the local Simulator, run XCUITest in CI on macOS runners, and farm out a small release-blocking suite plus exploratory manual passes to a real-device cloud. For a broader lens on cloud-phone tooling that includes Android, see our [best cloud phone emulators in-depth guide](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/). ## 11. Debugging Patterns That Save Days - **Reset before reproducing.** Most "works on my machine" iOS bugs are stale UserDefaults or Keychain state. `xcrun simctl erase all` first, reproduce second. - **Use the same Simulator family across the team.** A test that passes on iPhone 17 Pro and fails on iPhone SE is a real bug; a test that passes on your laptop and fails in CI because CI uses a different device is wasted hours. - **Always test the smallest supported width.** The cheapest source of layout bugs is the smallest iPhone in your support matrix; UI test that device first, not last. - **Reach for Instruments early.** Xcode 26's Processor Trace and Power Profiler instruments report per-subsystem energy and instruction-level traces; they catch animation jank and battery regressions that pure XCTest will miss. - **Snapshot tests over screenshot tests.** Pointfree's `swift-snapshot-testing`, run in the Simulator, gives diffable PNG and text snapshots in pull-request reviews. Fastlane `snapshot` is for marketing; `swift-snapshot-testing` is for engineering. - **Don't trust the Simulator for performance numbers.** Profile on the slowest device in your support matrix. Frame timing and memory pressure on a Mac M4 are not iPhone numbers. ## 12. Next Steps The Simulator is a fast feedback tool. It is not a substitute for a real iPhone, and it is not available outside macOS. Build the right pipeline: 1. Develop locally against the Xcode 26 Simulator. 2. Automate with `xcrun simctl` and fastlane. 3. Run CI on macOS Apple-Silicon runners. 4. Validate releases on a real-device cloud sized to your team. 5. Profile performance and battery on actual hardware. If the gap is people, not tools, [hire a Codersera-vetted iOS or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer). We pre-screen for Xcode and Simulator fluency, fastlane and CI experience, and shipping App Store releases under deadline pressure, so you skip the months it usually takes to find an engineer who is genuinely remote-ready and can extend your team without onboarding overhead. ## FAQ ### Is the iOS Simulator really not an emulator? Correct. It does not emulate iPhone hardware or boot a real iOS image. It loads iOS frameworks compiled for your Mac's CPU and runs your app as a native macOS process. ### Can I run the iOS Simulator on Windows or Linux? No. The Simulator depends on macOS-specific frameworks and Apple-redistributable binaries that ship only with Xcode. Anything claiming otherwise is either streaming a real Mac to your browser or isn't running iOS at all. ### What about a Hackintosh or a macOS VM on Windows? Technically possible, against Apple's licence terms (macOS is licensed only on Apple-branded hardware), and a fragile setup. Not recommended for a professional toolchain. Buy a Mac mini or use a cloud Mac like MacStadium or Scaleway Mac. ### Is Appetize.io a real iOS Simulator? Yes — it streams a real Xcode Simulator running on Apple-branded hardware in their cloud. It is not running on your local Windows machine; the pixels are coming over the wire. ### Can I submit an app to the App Store using only an online emulator? You still need an Apple Developer Program account and code-signing identities. You can do the development and submission from a streamed Mac (such as MacStadium) but not from any "iOS emulator for Windows" — those don't compile or sign iOS binaries. ### What's the difference between Simulator and a real iPhone for testing? Simulator covers UI, navigation, layout, accessibility, deep links, push payload formatting, and most logic. A real iPhone is required for camera, ARKit, CoreNFC, Bluetooth peripherals, Secure Enclave behaviour, real cellular conditions, real push delivery, and StoreKit live transactions. ### How do I test push notifications without a server? Drag an APNs JSON payload onto the Simulator window, or run `xcrun simctl push booted <bundle-id> payload.json`. The payload must include the `aps` dictionary and a `Simulator Target Bundle` key matching your app. ### How do I simulate a slow network in the Simulator? Install *Additional Tools for Xcode*, open Network Link Conditioner from System Settings, and choose a preset like 3G or "Very Bad Network." It applies to all network traffic on the host, including the Simulator. ### Is BrowserStack the same as Appetize.io? No. Appetize streams a Simulator. BrowserStack App Live streams a real iPhone. Higher fidelity, slower frame rate, more useful for release-blocking QA. ### What's the cheapest way to run iOS tests in CI? GitHub Actions `macos-latest` runners are included in the free tier for public repos and metered per-minute for private. For real-device coverage, AWS Device Farm at $0.17/device-minute or Firebase Test Lab at $5/hr are the lowest entry points. ### Can I run my app on iOS 17 if I only have Xcode 26? Yes. Older iOS Simulator runtimes are downloadable from *Xcode → Settings → Components*. You can keep iOS 17, 18, 26 runtimes side by side and target each from `xcodebuild`. ### Is Corellium an iOS emulator? Corellium virtualises iOS on ARM server hardware. It is closer to real iOS than the Xcode Simulator (it boots a real iOS kernel), but it does not have access to the Apple App Store, TestFlight, or App Store StoreKit. It is positioned for security research and reverse engineering, not for general-purpose product development. ### Does the Simulator support Apple Pay? Apple Pay is testable in a sandbox-only mode on the Simulator. Real production Apple Pay transactions require the Secure Enclave and only run on a physical device. ### Why does my app crash on a real iPhone but not in the Simulator? Common causes: differing CPU architectures revealing assumptions about pointer size or alignment; entitlements that only apply to signed builds; missing background-mode capabilities that the Simulator silently ignores; race conditions exposed only by slower device storage; or vendor SDKs whose Simulator slice is a no-op stub. ### Should I hire an iOS developer or learn this myself? If you're shipping a one-off internal tool, learn it. If iOS is a revenue surface, [hire a Codersera-vetted iOS or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer) and extend your team instead of bottlenecking on the toolchain. =============================================================== [19/20] Mobile App Testing in 2026: The Complete Guide URL: https://codersera.com/blog/mobile-app-testing-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== Mobile testing in 2026 is no longer a single discipline. It is a stack — unit tests on the JVM or a Swift target, integration tests against a mocked backend, instrumented UI tests on emulators, end-to-end flows on real devices in a cloud farm, and post-release crash and performance telemetry feeding the next sprint. Treat any one layer as the whole strategy and you will ship regressions. This guide covers how serious mobile teams structure that stack, what the 2026 tooling landscape looks like, what device clouds actually cost, and where the sharp edges are — written for engineers, QA leads, and tech leads making real build-vs-buy and in-house-vs-outsource calls, often alongside [Codersera-vetted mobile engineers](https://codersera.com/blog/hire/hire-react-native-developer) who have to live with the consequences. *Last updated: May 1, 2026.* ## TL;DR - The 70/20/10 testing pyramid (unit / integration / E2E) still holds for mobile, but device fragmentation pushes a thicker integration layer than on web. - Native frameworks — Espresso for Android, XCUITest for iOS — remain the gold standard for low-flake instrumented tests. Cross-platform work goes to Appium 2 (modular driver model), Maestro (YAML, black-box, <1% flake), or Detox (gray-box, React Native). - For Flutter, `integration_test` is the floor and Patrol is the practical ceiling — it crosses the native boundary that `integration_test` cannot. - Device clouds are not interchangeable. Firebase Test Lab is cheapest for short Android runs; AWS Device Farm wins on unmetered concurrency; BrowserStack and Sauce Labs lead on real-device breadth and enterprise features; LambdaTest competes on price; Kobiton focuses on session-based manual testing. - Crash and performance telemetry (Sentry, Firebase Crashlytics, Firebase Performance) is part of the test stack now — shift-right is how you cover the device matrix you cannot afford to test pre-release. - The right answer is almost always hybrid: emulators for unit and most UI tests in CI, a small real-device pool for nightly E2E, and a cloud farm for release-candidate matrix runs. ## What makes mobile testing structurally hard Most of the difficulty in mobile testing comes from things that simply do not exist on web. A web target is Chromium, Firefox, and WebKit on a handful of viewport sizes. A mobile target is two operating systems with multi-year version ranges still in active use, thousands of Android OEM SKUs with their own quirks, deep OS integrations (permissions, biometrics, push, deep links, background execution), and a network stack that ranges from gigabit Wi-Fi to a 3G cell on a train. Add the asynchronous nature of mobile UIs — animations, view recycling, threading models, and platform-imposed lifecycles — and "did the button do the thing" becomes non-trivial. Espresso and XCUITest exist primarily because Selenium-style polling does not work on a UI thread rendering at 120 Hz. Both ship idling resources or built-in synchronization; both still struggle with WebViews, custom Compose or SwiftUI components, and any animation that does not advertise its end state. If you are not yet comfortable with the emulator side of the equation, our [complete guide to Android emulators](https://codersera.com/blog/android-emulators-complete-guide-2026/) and the broader [survey of 32 mobile emulators](https://codersera.com/blog/32-best-mobile-phone-emulators-reviewed-in-2025-android-ios/) are useful background. ## The mobile testing pyramid in 2026 The classical Cohn pyramid — many fast unit tests, fewer integration tests, very few end-to-end tests — still maps cleanly onto mobile. The ratios most teams converge on are 60–70% unit, 15–25% integration, and 10–15% UI / E2E. Mobile tilts slightly more toward integration than web because so much of the value of a mobile app sits at the seam between your code and platform APIs (notifications, permissions, storage, background tasks). | Layer | What it covers | Where it runs | Typical tooling | Target run time | | --- | --- | --- | --- | --- | | Unit | Pure logic, view models, reducers, formatters | JVM (Android), Swift host process | JUnit, Kotest, XCTest, Quick / Nimble, Jest (RN) | < 5 ms / test | | Component / Widget | Single Compose / SwiftUI / RN / Flutter widget | JVM with Robolectric, host XCTest, Jest, flutter_test | Compose UI Test, ViewInspector, React Native Testing Library, flutter_test | < 100 ms / test | | Integration | Module + dependencies, mocked network, real DB | Emulator / simulator | AndroidX Test, XCTest, MockWebServer, Hilt / Koin test modules | 1–10 s / test | | UI / Instrumented | Single screen or short flow on device | Emulator (CI) and a few real devices | Espresso, XCUITest, Compose UI Test, EarlGrey 2 | 10–60 s / test | | E2E | Cross-screen user journeys, real backend or staging | Real devices, often via cloud | Maestro, Appium 2, Detox, Patrol | 1–5 min / flow | | Production telemetry | Crash, ANR, performance, regression detection | End-user devices | Sentry, Crashlytics, Firebase Performance | Continuous | "Shift-left" means push coverage into the bottom three rows, which run on every commit. "Shift-right" — staged rollouts, feature flags, crash telemetry — covers the device and locale matrix you cannot afford to enumerate pre-release. Both are necessary. ## Test types beyond functional Functional correctness is table stakes. The categories that distinguish a mature mobile test plan in 2026: - **Performance.** Cold start, time-to-first-frame, scroll jank, frozen frames. Android's Macrobenchmark library, Instruments on iOS, and Firebase Performance Monitoring in production cover this. - **Network conditions.** Test on simulated 3G, packet loss, and offline-then-reconnect transitions. BrowserStack, Sauce Labs, and most cloud farms expose network shaping. Locally, the Android emulator's network speed flag and Network Link Conditioner on macOS get most of the way there. - **Battery and thermal.** Background work that drains battery is a leading cause of one-star reviews. Android's Battery Historian and iOS's MetricKit are the tools of record. - **Accessibility.** TalkBack and VoiceOver flows, contrast, dynamic type, RTL. The Accessibility Scanner on Android and Accessibility Inspector on iOS catch the easy cases; manual screen-reader walkthroughs catch the rest. - **Security.** Static analysis (MobSF, Android Lint security checks), TLS pinning verification, root/jailbreak detection, OWASP MASVS coverage. - **Localization.** Long-string overflow, RTL mirroring, locale-specific date and number formats. Pseudo-localization in CI catches most truncation bugs before a translator ever sees the build. - **Beta and dogfood.** TestFlight on iOS, Play Console internal and closed testing on Android, plus Firebase App Distribution for ad-hoc cross-platform builds. ## The framework landscape The framework you pick determines what kind of flake you fight, how fast tests run, and how many engineers can read them. The 2026 shortlist: | Framework | Platforms | Approach | Language | Strengths | Tradeoffs | | --- | --- | --- | --- | --- | --- | | Espresso | Android | Gray-box, in-process | Kotlin / Java | UI-thread sync, low flake, Compose support | Android only; learning curve for IdlingResources | | XCUITest | iOS | Black-box, out-of-process | Swift | Apple-maintained, ships with Xcode, strong on simulators | iOS only; flakier on real devices than emulators | | EarlGrey 2 | iOS | White-box on top of XCUITest | Objective-C / Swift | Better synchronization than vanilla XCUITest | Small community outside Google; XCUITest is usually enough | | Appium 2 | iOS, Android, more | WebDriver, modular drivers | Any (JS, Java, Python, Ruby, C#) | Cross-platform, huge ecosystem, real and virtual devices | Slower than native; setup complexity; driver versions matter | | Maestro | iOS, Android, RN, Flutter, web | Black-box via accessibility layer | YAML | 10–15 min to first test, <1% flake, MaestroGPT for authoring | Less powerful for deep state assertions; YAML scales awkwardly past ~200 flows | | Detox | React Native (iOS, Android) | Gray-box, JS-thread aware | JavaScript / TypeScript | Idle-state synchronization, flake <2% on RN | RN-specific; 2–4 hour setup; brittle on native modules | | Flutter integration_test | Flutter (iOS, Android, web, desktop) | In-process via Flutter driver | Dart | Ships with Flutter SDK, fast, good widget control | Cannot drive native UI (system permissions, other apps) | | Patrol | Flutter (iOS, Android) | Wraps integration_test + native bridge | Dart | Drives native dialogs, permissions, Wi-Fi, biometrics | LeanCode-maintained; younger than integration_test | The practical 2026 default for a greenfield app: - **Native Android:** JUnit + MockK for unit, Compose UI Test + Espresso for instrumented, Maestro for E2E. - **Native iOS:** XCTest for unit, XCUITest for instrumented, Maestro or Appium for E2E if you need a single tool across platforms. - **React Native:** Jest + React Native Testing Library, Detox for deep RN E2E, Maestro for read-the-YAML-and-understand-it E2E. - **Flutter:** flutter_test, integration_test, Patrol for anything that crosses the native boundary. For React Native specifically, Detox's gray-box approach gives lower flake on the RN bridge but higher setup cost; Maestro's YAML brings time-to-first-test under fifteen minutes at the cost of less surgical control. Mature RN teams often run both. ## Devices: emulators, simulators, real devices, and the cloud Where a test runs is as important as how it is written. The four tiers, in increasing order of fidelity and cost: 1. **Local emulator (Android) or simulator (iOS).** Free, fast, scriptable. The iOS Simulator is genuinely close to a real device because it shares much of the underlying system; the Android Emulator with Google APIs is also strong but does not exercise OEM skin behavior. Most unit, component, and instrumented tests should run here. See our [Android emulators guide](https://codersera.com/blog/android-emulators-complete-guide-2026/) for a detailed comparison. 2. **Cloud emulator.** Same fidelity as local but parallelizable. Firebase Test Lab virtual devices, BrowserStack App Live virtuals, Genymotion Cloud. Useful for matrix runs without the local hardware bill. Our [cloud phone emulators guide](https://codersera.com/blog/best-cloud-phone-emulators-in-depth-guide/) goes deeper. 3. **Local real device.** A handful of "reference" devices — typically a current Pixel, a current iPhone, one mid-tier Android, and one older iPhone — wired to the workstation or to a self-hosted Bitrise / Codemagic agent. 4. **Cloud real-device farm.** Hundreds to thousands of physical devices in a data center, accessed by API or browser. Required for OEM-specific regressions, biometric flows, and any meaningful pre-release device matrix. For teams demoing without hardware, our roundups of [iPhone emulators for Windows](https://codersera.com/blog/best-iphone-emulator-for-windows-pc/), [iOS emulators for Mac](https://codersera.com/blog/best-ios-emulator-for-mac/), [virtual mobile device emulators](https://codersera.com/blog/best-virtual-mobile-device-emulators-run-android-and-ios-apps/), [free online iPhone emulators](https://codersera.com/blog/10-best-free-iphone-emulators-online-2025-no-download-required/), and [ApkOnline](https://codersera.com/blog/apkonline-android-emulator-a-comprehensive-guide/) separate legitimate options from snake oil. ## Device cloud comparison and pricing This is the table teams ask for and almost never find with real numbers in one place. All prices are public list pricing in May 2026 and round to the nearest sensible unit. Enterprise contracts are routinely 30–60% off list, and almost every vendor will negotiate. | Provider | Best for | Real / virtual | Pricing model | Entry price | Notes | | --- | --- | --- | --- | --- | --- | | Firebase Test Lab | Android matrix runs in CI | Both | Per device-hour, per-minute billing | $1/hr virtual, $5/hr physical (Blaze plan); free daily quotas on Spark | Cheapest for short Android runs; iOS support is limited. | | AWS Device Farm | Teams already on AWS, unmetered concurrency | Real (and remote access) | Per device-minute or unmetered slot | $0.17 / device-minute, or $250 / slot / month unmetered | Unmetered slots are the win — predictable cost at high volume. | | BrowserStack App Live | Manual, exploratory testing | Real | Per user / month | From ~$39 / user / month (annual) | Strong device breadth, geolocation, network sim. | | BrowserStack App Automate | Appium / XCUITest / Espresso CI | Real and virtual | Per parallel session | From ~$249 / month for App Automate Pro | Unlimited minutes; pay for parallels. | | Sauce Labs Real Device Cloud | Enterprise mobile + web combined | Real and virtual | Concurrency + minutes, annual | From ~$199 / month entry; enterprise commonly $20k–$75k+ / year | Real Device Access API (2026) for programmable infra. | | LambdaTest (TestMu AI) | Cost-conscious teams, web + mobile | Real and virtual | Per user / parallel | Real devices from $39 / month | Six product tracks; biometrics and camera injection included at entry. | | Kobiton | Manual + scriptless automation | Real | Minutes / month tiers | From $83 / month (500 min) to $399 / month (3000 min) | Strong on session-based manual testing and AI-assisted scripting. | | Codemagic / Bitrise | CI compute, not a device farm | Build agents | Per minute or seat | Codemagic from $0.095 / min macOS premium; Business $299 / month | Pair with Firebase Test Lab or BrowserStack for device coverage. | Two notes. First, "unlimited minutes" almost always means "limited parallels." Second, virtual-device cloud is only competitive with self-hosted CI emulators if your CI minutes are expensive (GitHub-hosted macOS) or your tests are slow to start. ## CI/CD integration The mobile CI pipeline in 2026 typically looks like this on every PR: lint and static analysis, unit tests, component tests, instrumented tests on a single emulator, build the debug APK / IPA. On merge to main: full instrumented matrix on Firebase Test Lab or BrowserStack, E2E on Maestro / Detox / Appium against a staging build, deploy to internal track and TestFlight. Nightly: full device matrix, performance benchmarks, security scans. The four CI choices most teams pick from: - **GitHub Actions.** Cheapest for Android; macOS minutes are 10× Linux minutes, which makes iOS painful at scale. Good for teams with light iOS volume. - **Bitrise.** Mobile-first, with prebuilt steps for Fastlane, code signing, Firebase Test Lab, App Store Connect. Stack stability is its main selling point. - **Codemagic.** Mobile-first, Flutter-native, pay-per-minute by default; Business plan at $299 / month gives unlimited macOS minutes for predictable spend. - **CircleCI.** Strong general-purpose CI with macOS support; better for teams that already have non-mobile workloads on it. Two rules apply regardless. Cache Gradle and Pods aggressively — half of any mobile pipeline's wall time is dependency resolution. And keep code signing off developer machines; Fastlane Match or your CI provider's managed signing is non-negotiable past three engineers. ## Crash, performance, and the shift-right side of the stack You cannot test every device-locale-OS combination pre-release. You can, however, observe what happens when real users hit the matrix. Crash and performance telemetry is now part of the test stack, not an afterthought. - **Firebase Crashlytics.** Free, deep Firebase integration, groups crashes by stack trace. Strong default for Android-led teams; iOS support is solid. - **Sentry.** Cross-platform (mobile, web, backend), per-event detail rather than aggregation, release-health metrics, performance tracing. Modern SDKs add roughly 1% CPU overhead. Better when the same team owns mobile and backend. - **Firebase Performance Monitoring.** App start time, network request latency, custom traces. Pairs with Crashlytics. - **Play Console / App Store Connect vitals.** ANRs, excessive wakeups, crash-free user rate. Free, authoritative, often the first place a regression shows up. The pattern that works: gate releases on crash-free-user-rate thresholds (typically 99.5%+ paid, 99%+ free) and tie staged rollouts to those gates. A rollout that auto-pauses on regression is worth more than another hundred E2E tests. ## Cost reality and when to outsource A realistic 2026 mobile test budget for a mid-size product team: - CI compute: $300–$2,000 / month depending on iOS volume. - Device cloud: $500–$5,000 / month for one app. - Crash and performance telemetry: $0 (Crashlytics) to $2,000 / month (Sentry at scale). - Local device lab: a few thousand dollars one-off, plus $200–$500 / month maintenance. - QA headcount: one QA engineer per three to five mobile engineers. Build-vs-buy decisions worth thinking through: - **Self-hosted lab vs cloud farm.** Below ~50 daily runs, cloud wins on TCO. Above that, a small in-house lab pays back inside a year — but only if someone owns it. - **In-house automation vs outsourced QA.** Outsource regression and exploratory testing on stable features. Keep framework ownership and CI in-house — that is where knowledge compounds. - **Generalist engineers vs specialist SDETs.** Up to ten engineers, generalists work. Past that, a dedicated mobile SDET role pays for itself. ## Known issues and sharp edges - **Compose and SwiftUI flakiness.** Both modern UI toolkits sometimes confuse the underlying test frameworks' idle detection. Animations that loop forever or use spring physics are the most common offenders. Disable animations in test builds. - **WebViews.** Espresso and XCUITest both treat WebViews as a black box. You either drop into Espresso-Web / WKWebView APIs or accept that those flows go to E2E tools like Appium. - **Permissions and system dialogs.** Anything that pops the OS-level permission sheet breaks pure-Flutter, pure-RN tools. Patrol (Flutter), Maestro, and Appium can drive those dialogs; integration_test and Detox cannot. - **Real-device flake.** Real iPhones in cloud farms are noticeably flakier than simulators because they share devices across tenants, get rebooted between sessions, and occasionally lose Wi-Fi. Plan for retries; do not gate every PR on real-device E2E. - **Native module upgrades.** A React Native or Flutter version bump frequently breaks Detox or Patrol. Pin versions and treat the bump as a project, not a chore. - **Code signing.** The most common reason a pipeline goes red is an expired profile. Automate via Fastlane Match. - **Cloud-farm queueing.** Specific models have queues at peak hours; pin to a device family, not a model. - **Test data.** Tests sharing a staging account fight each other. Provision per-test users or use seeded fixtures. ## FAQ ### What is the difference between mobile testing and web testing? Mobile testing has to deal with multiple operating systems, hundreds of OEM device variations, deep platform integrations, variable network conditions, and battery and thermal constraints. Web testing is mostly three browser engines and a handful of viewport sizes. The mobile testing pyramid therefore tends to be flatter, with relatively more integration and device-level testing. ### Should we use Espresso and XCUITest, or a cross-platform tool? For instrumented tests on a single platform, native frameworks are faster and less flaky. For flows that need to behave identically on both platforms, a cross-platform tool (Maestro, Appium 2) reduces duplication. Most mature teams use both. ### Is Appium 2 still relevant in 2026? Yes. The modular driver model decoupled the core server from platform drivers, making it lighter and easier to scale in containers. It remains the most flexible option when you need to drive iOS, Android, and other targets from a single suite in any major language. ### Maestro or Detox for React Native? Detox if you want gray-box JS-thread synchronization and your engineers will own the suite — flake under 2%, setup time 2–4 hours. Maestro if QA or product will help author flows — YAML, time-to-first-test under 15 minutes, flake under 1%. Many teams use both. ### What is the difference between integration_test and Patrol for Flutter? `integration_test` ships with Flutter and can drive widgets in the app's own tree. Patrol wraps `integration_test` and adds a native bridge so your tests can also tap system permission dialogs, toggle Wi-Fi, drive biometrics, and interact with other apps. Use Patrol whenever your test crosses the native boundary. ### How many real devices do we actually need? A defensible local matrix is one current and one previous iPhone, one current and one budget Android, plus whichever device represents your largest user segment in production. Anything beyond that should live in a cloud farm. Look at your Crashlytics or Sentry device breakdown — five devices typically cover 60–70% of your real users. ### Which device cloud is cheapest? Firebase Test Lab is cheapest for short Android runs ($1 / hr virtual, $5 / hr physical, with free daily quotas). LambdaTest is the cheapest entry point for real devices in a self-serve plan ($39 / month). AWS Device Farm wins when you need predictable cost at high volume thanks to its $250 / slot / month unmetered option. ### Can we replace E2E tests with crash analytics? No, but they cover different gaps. E2E tests catch regressions in flows you specifically wrote tests for. Crash analytics catches regressions in flows you did not anticipate, on devices and OS versions you did not test. You need both. ### How do we keep flaky tests under control? Three habits. Quarantine flaky tests in a separate suite that does not block merges, but track time-in-quarantine and treat it as tech debt. Use frameworks with built-in synchronization (Espresso, XCUITest, Maestro, Detox) instead of `sleep()` calls. Disable animations in test builds, and prefer test ID accessibility identifiers over text-based locators. ### Should we test on iOS Simulator or real iPhones? Both. The Simulator is fine for unit, component, and most XCUITest runs. Real iPhones catch issues that only show up on hardware: camera, biometrics, Bluetooth, push, thermal performance, and UIKit edge cases. Run real-device tests nightly and on release candidates. ### What about manual testing — is it dead? No. Exploratory manual testing finds bugs no automated suite will. The 2026 shift is toward making manual testing exploratory rather than scripted: anything you would write a script for, automate. ### How do we test for a region we have no devices in? Use a cloud farm with regional devices for matrix runs, and lean on Crashlytics or Sentry breakdowns by country, locale, and carrier. Simulate that region's network conditions in CI too — a flaky 3G connection in Lagos behaves nothing like Wi-Fi in Mountain View. ### What is "shift-left" vs "shift-right" in mobile testing? Shift-left moves testing earlier — unit, component, static analysis, contract — so regressions are caught at commit time. Shift-right pushes observation into production: staged rollouts, feature flags, crash and performance telemetry. Shift-left covers what you know to test; shift-right covers what you did not. ### In-house QA or outsourced QA? Outsource what is repetitive and stable: regression runs, exploratory testing of mature features, localization. Keep in-house anything that compounds knowledge: framework ownership, CI maintenance, performance benchmarking, per-feature test planning. Outsourcing the framework itself freezes it at the contractor's day-one skill level. ## Next steps Building from scratch, start at the bottom: get unit and component coverage above 60% before investing in E2E. Scaling an existing strategy, audit flake rate, CI wall time, and crash-free user rate — those three numbers tell you where the next dollar goes. Hiring for any of this, the bottleneck is rarely "knows Espresso" — it is engineers who reason about framework, CI, device strategy, and production telemetry as one system. [Hire a Codersera-vetted mobile or React Native engineer](https://codersera.com/blog/hire/hire-react-native-developer) when you need someone who has shipped this end-to-end before, not just written tests against a tutorial app. =============================================================== [20/20] Software Testing in 2026: The Complete Engineer's Guide URL: https://codersera.com/blog/software-testing-complete-guide-2026/ Last updated: 2026-05-01 =============================================================== *Last updated: May 1, 2026.* Testing is the part of the job most engineers say they care about and quietly skip when deadlines tighten. That gap is the whole problem. In 2026, with AI agents committing code faster than humans can review, testing is a load-bearing part of the engineer's job description — not a QA team's chore. This guide is for the engineer who owns quality: writes the code, writes the tests, and is paged at 3 a.m. when both fail. We cover what is real now — the testing trophy versus the pyramid, where snapshot tests still earn their keep, the AI-augmented testing wave, the specific frameworks worth your time per ecosystem, flaky test management, performance and chaos testing, ephemeral environments, and when staffing a dedicated SDET pays off. ## TL;DR - **Forget the pyramid as dogma.** Use Kent C. Dodds' testing trophy for frontends and integration-heavy services; keep a pyramid for backends with deep business logic. The right shape depends on what your code is mostly doing — gluing systems together or computing things. - **Coverage is a vanity metric on its own.** 80% line coverage with weak assertions catches almost nothing. Pair coverage with mutation testing (Stryker, PIT, cargo-mutants) before you trust a number. - **Playwright won.** 33M weekly npm downloads, 45% adoption among QA pros in 2026, real WebKit support, free parallelization. Cypress is fine if you already run it; do not start a new project on it. - **Vitest is the default for new JS/TS projects.** 5x faster cold starts and ~28x faster watch reruns than Jest 30. Angular 21 made it the default. Stay on Jest only if migrating costs more than it earns. - **AI-generated tests are real but dangerous.** Claude Code and Cursor write plausible tests fast; they also write tests that pass without asserting anything meaningful. Mutation testing is now table stakes for AI-written suites. - **Flaky tests cost ~6–8 hours per engineer per week.** Quarantine them, do not retry them blindly. Test impact analysis (Datadog, Bazel, Nx) is the highest-ROI CI investment most teams skip. - **Hire engineers who treat testing as part of the job** before you hire a dedicated QA team. [Codersera-vetted engineers](https://codersera.com/blog/hire) ship code with tests, not behind them. ## Testing Fundamentals: Pyramid, Trophy, and What Actually Matters in 2026 The test pyramid — many unit tests, fewer integration, very few end-to-end — was Mike Cohn's 2009 model and dominated thinking for over a decade. It is still right for some workloads. It is wrong for many others. Kent C. Dodds proposed the testing trophy in 2018: static analysis at the base, then unit, then a fat middle layer of integration tests, then a thin layer of E2E. The argument is simple and increasingly true: most modern applications are integration code. A React component that renders a list, fetches data, handles errors, and writes back to a store is not a unit — it is a small system. Test it as a small system or your tests are not testing what users actually do. The 2026 reality is that both shapes are valid and the choice should be deliberate: - **Backend services with heavy domain logic** — pricing engines, scheduling, fraud scoring — earn the pyramid. Pure functions deserve isolated, fast unit tests. - **Frontends, BFFs, and orchestration services** — anything where the value is in how pieces connect — earn the trophy. Integration tests with realistic boundaries (real DOM, real DB, mocked third-party APIs) catch the bugs your users hit. - **Static analysis is non-negotiable.** TypeScript strict mode, ESLint with `typescript-eslint`, Ruff or Pyright for Python, `golangci-lint` for Go. Cheap tests that run on save eliminate entire bug classes before you write a single assertion. If your team is religiously chasing 80% unit coverage on glue code that mostly calls other functions, you are writing the wrong tests. If you have one giant Cypress suite and no isolated tests for a regex-heavy parser, same problem. ## Types of Tests and When Each One Earns Its Keep | Type | What it tests | Use it when | Skip it when | | --- | --- | --- | --- | | Unit | One function/class in isolation | Pure logic, algorithms, parsers, calculators | Code is mostly orchestration with few branches | | Integration | Multiple modules wired together, real DB or in-memory equivalent | API handlers, React components with state, service layers | You can get the same confidence from a unit test | | Contract | Producer and consumer agree on a schema (Pact, OpenAPI diff) | Microservices, public APIs, separate teams | Monolith with a single team | | End-to-end (E2E) | Full user flow through a real browser/app | Smoke-test the critical path: signup, checkout, search | You are tempted to test every edge case here — those belong lower | | Smoke | "Did the deploy come up?" — a tiny critical-path E2E | Every deploy, every environment | Never skip | | Regression | Old bugs do not come back | Add one whenever you fix a bug | You did not write a failing test before the fix | | Snapshot | Output matches a stored fixture | Stable, deterministic output: serializers, generators, public DTOs | UI components — they churn and snapshots become rubber-stamps | | Mutation | Your tests fail when code is deliberately broken | Critical modules, AI-generated test suites | You cannot afford 10–60x runtime — run nightly instead | | Property-based | Invariants hold for generated inputs | Parsers, sorts, encoders, anything with mathematical properties | Pure UI work | | Fuzz | Random/malformed input does not crash | Anything that parses untrusted bytes | Internal-only typed APIs | | Chaos | System survives infrastructure failures | Distributed systems with SLOs | Single-binary apps with no dependencies | | Load/performance | Latency and throughput under concurrent load | Anything user-facing with a perf SLO | Internal batch jobs measured by wall clock | | Security | OWASP Top 10, dependency CVEs, SAST/DAST | Always | Never — bake into CI | | Accessibility | WCAG compliance — axe-core in CI | Any consumer-facing UI | Internal admin tools (still nice to have) | Two opinionated calls: **Snapshot tests are dead for UI components, alive for serializers.** Jest snapshots locking your React tree produced a generation of engineers who hit "u" to update without reading the diff — a rubber stamp, not a test. Snapshots still work where the diff is small and meaningful: GraphQL schemas, generated SDKs, API response DTOs. **Coverage targets above 80% are mostly cargo-culting.** Going from 80% to 95% is expensive and gains little unless paired with mutation testing. ThoughtWorks' April 2026 Radar (Vol. 34) flags mutation testing — Stryker, PIT, cargo-mutants — as the way to "shift focus from how much code is executed to how much code is actually verified." ## Frameworks Per Ecosystem Pick boring tools your team already knows over novel tools nobody knows. | Ecosystem | Unit/Integration | E2E | Mutation | Property-based | Performance | | --- | --- | --- | --- | --- | --- | | JavaScript/TypeScript | Vitest (default), Jest 30 (legacy) | Playwright | Stryker | fast-check | k6, Artillery | | Python | Pytest | Playwright (Python bindings) | mutmut, Cosmic Ray | Hypothesis | Locust, k6 | | Go | `go test` + testify | Playwright via testcontainers | go-mutesting | gopter, native fuzz (`go test -fuzz`) | k6, vegeta | | Java/Kotlin | JUnit 5 + AssertJ | Playwright (Java) or Selenium | PIT (PITest) | jqwik | Gatling, JMeter, k6 | | Ruby | RSpec, Minitest | Capybara + Playwright | Mutant | Rantly | k6, JMeter | | Rust | Built-in `cargo test`, rstest | Playwright via webdriver | cargo-mutants (Trial on TW Radar) | proptest, quickcheck | criterion (microbench), k6 | | .NET | xUnit, NUnit | Playwright (.NET) | Stryker.NET | FsCheck | NBomber, k6 | **Vitest vs Jest:** for new projects, Vitest. 2026 benchmarks show Vitest 2.0 finishing 10,000 React component tests ~3.8x faster than Jest 30 with ~40% lower memory overhead, and watch reruns in hundreds of milliseconds instead of seconds. Vitest crossed 40M weekly downloads while Jest plateaued near 36M; Angular 21 made it default. Migration is mostly mechanical. Stay on Jest only if your suite is already fast enough. **Playwright vs Cypress:** Playwright. It overtook Cypress on every metric — 33M vs 6.5M weekly npm downloads, real WebKit support, free built-in sharding, ~290ms per action vs Cypress's ~420ms. Cypress's time-travel debugger is still the best in class, but not worth giving up cross-browser coverage and free parallelization. Healthy Cypress suites do not need to panic-migrate; new projects should pick Playwright. For Python, **Pytest** remains uncontested. For Go, the standard library plus `testify` covers ~95% of needs and Go 1.18+ ships a real fuzzer. For Java, JUnit 5 with AssertJ and Testcontainers is canonical. ## The AI-Augmented Testing Wave Two distinct things are happening, and they get conflated constantly. **First: AI generating unit and integration tests inside your IDE.** Cursor, Claude Code, and the broader agent crop write plausible tests against your codebase in seconds. Done well, this is useful — Claude Code reasons about test patterns across the whole repo and writes consistent fixtures. Done badly, you get tests that pass without asserting anything meaningful: an LLM that mocks the function under test, then asserts the mock was called. The mitigation is mutation testing on AI-generated suites and human review focused on the assertions. See our deep-dives on [AI coding agents](https://codersera.com/blog/ai-coding-agents-complete-guide-2026/), [Cursor](https://codersera.com/blog/cursor-ide-complete-guide-2026/), and [Claude Opus 4.7](https://codersera.com/blog/claude-opus-4-7-complete-guide-2026/). **Second: AI-native test platforms — Mabl, Functionize, ProductScript, QA Wolf, Autonoma.** These record or describe a flow in natural language, then maintain locators when the UI shifts. The State of Testing 2026 report shows AI-augmented tools delivering a 12.1% increase in automation coverage, ~10.8% drop in production defects, and 40–45% maintenance cost reductions on self-healing suites. Real numbers — but achieved by mature teams. Bolting Mabl onto a chaotic codebase will not save it. Rough 2026 pricing: **Mabl** ~$40k–$80k/year; **Functionize** enterprise from $50k+; **QA Wolf** managed service $5k–$30k/month; **ProductScript** and similar AI-agent tools $200–$2,000/month per team. Decision rule: if your engineers will write and maintain tests, stay on open-source (Playwright + Vitest/Pytest) and let AI assist in the IDE. If they will not write tests under any circumstances, an AI-native platform beats the zero-test status quo — but it is a stopgap, not a strategy. ## CI/CD Test Orchestration The biggest CI improvement most teams skip is **test impact analysis** — only running tests affected by the diff. Datadog Test Optimization, Bazel, Nx, and Turborepo all do this. Faster PRs, less flake exposure, lower CI bills. A working 2026 CI pipeline for a typical web app: 1. **On every push:** lint, typecheck, unit tests for changed modules (test impact analysis), security scans (Snyk, Dependabot, Trivy for containers). 2. **On PR:** integration tests, contract tests against a shared mock provider (Pact Broker / PactFlow), accessibility scan with axe-core, full Playwright suite sharded across 4–8 workers. 3. **On merge to main:** deploy to a staging or ephemeral preview environment, run smoke E2E, gate on a k6 or Artillery perf check that asserts p95 latency hasn't regressed beyond a threshold. 4. **Nightly:** mutation testing run, full E2E across all browsers, dependency audit, chaos experiments in non-prod. Run the slow tests where they belong — at night, not blocking PRs. A 45-minute PR pipeline trains the team to skip tests; a 6-minute PR pipeline trains the team to write more. ## Flaky Tests: Quarantine, Do Not Retry Flaky tests cost the average engineering team 6–8 hours per engineer per week, according to Datadog's 2026 telemetry. The wrong response is automatic retries. Retries hide real bugs, normalize unreliability, and exhaust CI budget. The right response is a quarantine workflow: 1. Detect flakes statistically — same commit, both pass and fail across runs. Datadog, Trunk.io, BuildPulse, and CircleCI all surface this now. 2. Auto-quarantine flaky tests so they stop blocking CI but still report. 3. File a ticket with full context and an owner. 4. Set a hard SLA — a quarantined test that is not fixed in two weeks gets deleted, not ignored. Datadog's Bits AI Dev Agent now auto-generates fixes for detected flakes as PRs; we have seen it produce solid fixes for race conditions and selector instability and weak fixes that just paper over async timing. Treat those PRs like any AI-generated PR: the diff matters, the green check does not. ## Performance Testing The four mainstream tools: - **k6:** Go binary, JS scripting, lowest CPU/memory per VU, best CI/CD integration. Free; Grafana Cloud k6 from ~$30/month. - **Locust:** Python-native, real-time UI, friendly to AI-assisted script generation. Pick this for Python-first teams. Free. - **JMeter:** broadest protocol support (JDBC, LDAP, JMS, SMTP). XML plans fight version control; for non-HTTP protocols nothing else competes. Free. - **Artillery:** YAML-driven; GraphQL, gRPC, WebSockets, Kafka, Playwright browser load. Free core, paid cloud. Default to k6 for HTTP/gRPC perf in CI, Locust for Python shops, JMeter only when you need its protocols. Run perf tests against ephemeral envs, not shared staging. ## Test Environments and Data The 2026 default is **ephemeral preview environments** — a full stack spun up per PR, torn down on merge. Vercel and Netlify ship this for static/BFF workloads; Bunnyshell, Northflank, Shipyard, and Tilt's ephemerator handle Kubernetes. Garden remains strongest for declarative multi-service envs wired into local dev. Shared staging is a tragedy of the commons — conflicting changes, drifting data, "is this broken?" Slack threads as the actual gate. Ephemeral envs cost more in cloud bill and pay back in test reliability. Test data is the other half. Three patterns that work: - **Builders/factories** (factory-bot, fishery, polyfactory) for unit and integration tests — readable, composable, no test pollution. - **Sanitized production snapshots** for performance and integration testing — Tonic, Snaplet, Neosync, or in-house pipelines. Real shapes, no real PII. - **Reset-per-test database transactions** for integration tests against a real DB — Postgres `TRUNCATE` is fine; transactional rollback is faster but fights with code that opens its own transactions. ## Shift-Left, Shift-Right, and Observability-as-Testing Shift-left — testing earlier in the cycle — has been the consensus for a decade. The newer movement is shift-right: testing in production with feature flags, canary deploys, synthetic monitors, and structured observability. Datadog, Honeycomb, and Sentry have collapsed the line between "test" and "monitor." If your synthetic checks fire k6 scripts every five minutes against production with budgeted error rates, that is testing. If your deploys are staged behind feature flags with automatic rollback on error-rate regression, that is testing. The 2026 reality: fewer pre-prod tests, more guardrails around production. The marginal hour spent on a 99th E2E test is usually worse-spent than wiring up a synthetic check, SLO burn-rate alert, or feature-flag rollback. Chaos engineering belongs here too. Tools to know: - **LitmusChaos** — CNCF, Kubernetes-native, the open-source default. - **Gremlin** — commercial, polished UI, the safest bet for first-time chaos. - **Chaos Mesh** — CNCF, strong network and IO fault injection. - **AWS Fault Injection Service** — if you live in AWS, the lowest-friction option. Chaos Monkey itself is mostly historical now — useful as a reference, not a tool you would deploy fresh in 2026. ## When to Outsource QA, When to Hire an SDET, When to Just Hire Better Engineers The honest answer most engineering leaders do not want to hear: most teams asking "should we hire a QA team?" should instead hire engineers who write tests as part of shipping. A separate QA team becomes a wall to throw quality over, and engineering's testing skill atrophies. Where dedicated QA capacity does make sense: - **Regulated industries** — finance, healthcare, automotive — where audit trails and signed-off test plans are mandatory. - **Complex hardware/software products** with lab environments or specialized devices (see our [Android emulators guide](https://codersera.com/blog/android-emulators-complete-guide-2026/)). - **Mature products at scale** where one SDET owns E2E infrastructure, flake triage, and test tooling. - **Outsourced QA pods** for surge work — launches, large UI overhauls — never as a permanent crutch. For most early- and mid-stage teams, the highest-leverage move is hiring engineers whose definition of "done" includes tests, observability, and deploy safety. That is what Codersera screens for. ## Sharp Edges and Common Mistakes - **Mocking the thing you are testing.** If your test mocks the function under test and asserts the mock was called, you have written a tautology. Common with AI-generated tests; catch it with mutation testing or careful review. - **Coverage as a goal, not a signal.** 90% line coverage with weak assertions is worse than 60% with strong ones — it manufactures false confidence. Pair coverage with mutation scoring before celebrating. - **One giant E2E suite as the only test.** Slow, flaky, expensive, and tells you "something broke" without telling you what. Push tests down the pyramid/trophy. - **Auto-retrying flaky tests.** Hides real concurrency bugs and trains the team that green CI does not mean working code. - **Snapshot-spamming UI tests.** Updating snapshots without reading the diff is not a test, it is a habit. Use snapshots only where the diff is small and meaningful. - **Performance tests against shared staging.** Misleading results, angry teammates. Spin up an ephemeral env per perf run. - **Skipping accessibility.** axe-core in CI takes one afternoon to set up and catches most WCAG issues. ThoughtWorks moved it to "Adopt" for a reason. - **Mutation testing on the wrong scope.** 10–60x slower than your test suite; run on critical modules nightly, not on every PR. - **Treating tests as second-class code.** Test code with no review, no refactoring, and no DRYing decays into the worst code in your repo. Review tests like production code. - **Letting AI-generated tests through without review.** They look right, they run green, and they assert nothing. The 30 seconds of review you skip costs you a week of debugging later. ## FAQ ### Is the test pyramid dead? No. It is just no longer universal. For backend services with rich domain logic, the pyramid is still right. For frontends and orchestration code, the testing trophy is the better mental model. Pick deliberately. ### What code coverage target should we hit? For most production code, 70–80% line coverage with strong assertions and a mutation score above 60% is solid. Above 90% line coverage typically signals diminishing returns and brittle tests. Critical modules — auth, billing, anything money-touching — earn 90%+ and a mutation score above 80%. ### Should we migrate from Jest to Vitest? For new projects, yes — Vitest by default. For existing Jest projects, migrate when the suite is slow enough that engineers avoid running it locally. Mechanical migrations of mid-size suites usually take 1–3 days; transformer-heavy projects take longer. ### Should we migrate from Cypress to Playwright? Only if you need cross-browser (Safari/WebKit) coverage, hit Cypress Cloud parallelization paywalls, or are starting fresh. Healthy Cypress suites are not worth disrupting just for the badge. ### Are AI-generated tests safe to merge? Only after human review focused on the assertions. The structure looks right by default; the assertions are where AI-written tests fail silently. Run mutation testing on suites that include heavy AI contributions. ### How do we deal with flaky tests? Detect statistically, quarantine fast, file a ticket with an owner, hard SLA on fixing or deleting. Never auto-retry blindly. ### Is contract testing worth setting up? Yes, if you have multiple services owned by different teams and you want to retire most cross-service E2E tests. Pact with PactFlow is the canonical setup; budget two weeks for the first integration. ### Is mutation testing too slow to use? For full-suite use, yes — 10–60x slower than your tests. Scope it: nightly runs on critical modules, or on PRs that touch high-stakes code paths. Stryker, PIT, and cargo-mutants all support module-level scoping. ### k6 or Locust for performance testing? k6 if you want lower resource use and the best CI integration. Locust if your team is Python-first and you want to share helpers with your app. Both are excellent. ### Should we run chaos engineering experiments in production? Eventually, yes — Netflix's whole point. Start in staging or ephemeral envs with LitmusChaos or Gremlin, build confidence, then graduate to production with strict blast-radius controls and clear stop conditions. ### What is observability-as-testing? The idea that production telemetry — error rates, SLOs, synthetic monitors, feature-flag-gated rollouts with automatic rollback — does some of the work that pre-prod tests used to. Not a replacement for tests, but a complement that catches what pre-prod cannot. ### When does it make sense to hire a dedicated SDET? When test infrastructure (E2E platforms, flake triage, perf harnesses, environment management) is full-time work, typically around the 8–15 engineer mark for product teams, or earlier for regulated and high-uptime products. ### Should we outsource QA? For surge work — launches, large UI overhauls, regression hardening — yes, an outsourced pod can be efficient. As a permanent function for an engineering team that should be writing tests itself, no. It tends to ossify the "engineering writes code, QA finds bugs" anti-pattern. ### What is the single highest-ROI testing investment for a 10-engineer team in 2026? Test impact analysis plus ephemeral preview environments. Together they cut PR-cycle time, reduce flake exposure, and let real integration tests run on every change without team contention. ## Next Steps Testing is not a phase of the SDLC, it is part of the discipline of writing software. The teams shipping reliably in 2026 are the ones whose engineers internalized that — they pick the right test type for the job, they treat AI-generated tests with healthy suspicion, they invest in CI infrastructure (test impact analysis, ephemeral envs, flake quarantine) before they invest in headcount, and they treat production observability as part of the test loop. If you are hiring engineers and the candidates you are seeing treat testing as someone else's job, you have a hiring problem, not a process problem. [Hire a Codersera-vetted engineer who treats testing as part of the job](https://codersera.com/blog/hire) — and keep the QA-team-shaped hole on your org chart empty until you actually need it.