How to Run SAM 3 Locally: 2026 Install Guide

Meta released SAM 3 (Segment Anything Model 3) on 19 November 2025, and shipped a drop-in SAM 3.1 update in March 2026. Unlike SAM 2, which needed you to click a point or draw a box for every object, SAM 3 introduces promptable concept segmentation: you type a short noun phrase like "yellow school bus" and it finds, segments, and tracks every matching instance across an image or video. This guide walks through a real local install on Linux/CUDA, the Hugging Face access gotcha that trips up most people, and honest expectations for Apple Silicon and CPU.

What can SAM 3 actually do?

SAM 3 is an ~840M-parameter model (the checkpoint weighs roughly 3.4 GB) that unifies detection, segmentation, and tracking behind a single prompt interface. It accepts three kinds of prompts:

• Text (open-vocabulary) prompts — short noun phrases. It returns a mask for every instance of that concept, not just one object. This is the headline feature.
• Image exemplar prompts — draw a box around one example object and SAM 3 finds visually similar instances.
• Point and box prompts — full backward compatibility with SAM 2 for interactive single-object masking.

Architecturally it pairs a shared vision encoder with a DETR-style detector and a memory-based tracker, plus a presence head that separates the "what" (recognition) from the "where" (localization). That is what lets it tell apart close prompts like "a player in white" versus "a player in red." It was trained with a data engine that annotated over 4 million unique concepts, and Meta shipped the SA-Co benchmark (120K images, 1.7K videos, 200K+ concepts) alongside it. On the LVIS zero-shot mask benchmark it reports around 47.0 AP.

What do you need to run SAM 3 locally?

SAM 3's official code path assumes an NVIDIA GPU. There is no first-class CPU or Apple Silicon build (more on that below). The baseline requirements:

How do you install SAM 3 on Linux with CUDA?

Use a clean conda environment so PyTorch and CUDA don't collide with other projects. These are the exact steps from Meta's repository:

conda create -n sam3 python=3.12 -y
conda activate sam3

# Install a CUDA build of PyTorch first
pip install torch==2.10.0 torchvision --index-url https://download.pytorch.org/whl/cu128

# Clone and install SAM 3
git clone https://github.com/facebookresearch/sam3.git
cd sam3
pip install -e .

# Optional: example notebooks
pip install -e ".[notebooks]"

If you want the fastest inference, the repo also supports FlashAttention-3 and a custom CUDA NMS kernel. These are optional and only help on capable GPUs:

pip install einops ninja
pip install flash-attn-3 --no-deps --index-url https://download.pytorch.org/whl/cu128
pip install git+https://github.com/ronghanghu/cc_torch.git

The checkpoint access gotcha (read this before you debug for an hour)

SAM 3's weights are gated. If you try to load the model without authenticating you'll get a 401/403 from Hugging Face, not a clear "you need access" message. Do this first:

1. Open huggingface.co/facebook/sam3 (or facebook/sam3.1 for the latest) and click Request access. Approval is usually quick but not instant.
2. Create a token at huggingface.co/settings/tokens (read scope is enough).
3. Authenticate the CLI so downloads are authorized:

pip install -U huggingface_hub
hf auth login    # older CLIs: huggingface-cli login
# paste your token when prompted

Once your account is approved and the CLI is logged in, the model builder downloads the checkpoint automatically the first time you construct the model (load_from_HF=True is the default).

How do you run text-prompt segmentation?

Here is the minimal image example using SAM 3's native processor. The set_image() call runs the encoder once; you can then reuse that state across many text prompts cheaply.

from PIL import Image
from sam3.model_builder import build_sam3_image_model
from sam3.model.sam3_image_processor import Sam3Processor

model = build_sam3_image_model()          # auto-downloads the gated checkpoint
processor = Sam3Processor(model)

image = Image.open("street.jpg")
state = processor.set_image(image)

output = processor.set_text_prompt(state=state, prompt="yellow school bus")
masks  = output["masks"]                  # one mask per detected instance
boxes  = output["boxes"]
scores = output["scores"]
print(f"found {len(masks)} instances")

Point and box prompts

If you want SAM 2-style single-object masking (click or box), the simplest route is the Ultralytics wrapper, which loads the same sam3.pt checkpoint and exposes a compact API:

pip install ultralytics

from ultralytics import SAM

model = SAM("sam3.pt")
# box prompt (xyxy pixel coords) -> segments the one object inside the box
model("street.jpg", bboxes=[100, 150, 400, 500])
# point prompt -> label 1 = foreground
model("street.jpg", points=[300, 250], labels=[1])

How do you track objects in a video?

Video uses a session-based predictor: start a session on the clip, add a text (or box) prompt on a frame, and SAM 3 propagates masks forward and backward, keeping consistent identities.

from sam3.model_builder import build_sam3_video_predictor

predictor = build_sam3_video_predictor()
resp = predictor.handle_request(
    request=dict(type="start_session", resource_path="clip.mp4")
)
resp = predictor.handle_request(
    request=dict(
        type="add_prompt",
        session_id=resp["session_id"],
        frame_index=0,
        text="person",
    )
)

SAM 3.1 is a drop-in checkpoint swap here. Its object-multiplexing change tracks up to 16 objects in a single forward pass and roughly doubles throughput for medium-object scenes, plus adds "global reasoning" so objects that briefly leave the frame get re-identified correctly. If you're doing multi-object video, prefer facebook/sam3.1.

Can you run SAM 3 on a Mac (Apple Silicon / MPS) or CPU?

Honestly: not well, and not with the official repo as of mid-2026. SAM 3's code assumes CUDA throughout, and its performance helpers (custom NMS, Triton kernels) have no CPU/MPS fallback. Running on Apple Silicon through PyTorch's mps backend currently fails on a pin_memory() path that behaves differently than on CUDA. Backend-agnostic inference is an open request upstream, not a shipped feature.

Practical options if you don't have an NVIDIA GPU:

• Ultralytics on CPU — SAM("sam3.pt") with device="cpu" can run image inference on a Mac. Expect it to be very slow (seconds per image), and treat it as a correctness sandbox, not a workflow.
• Rent a CUDA GPU — a single mid-range cloud GPU is the path of least resistance for real work. Many teams would rather bring in engineers who already have a working GPU/MLOps pipeline than fight local drivers.
• The hosted Segment Anything Playground — Meta's browser demo lets you try text and box prompts with zero install before you commit to a local build.

What VRAM and speed should you expect?

SAM 3 is heavier than SAM 2 by design — it's doing detection plus segmentation plus tracking. Rough, real-world expectations:

• Images: single-image inference sits comfortably on a 12 GB card; batching many images pushes VRAM toward 18–22 GB.
• Video is the bottleneck. Community testing on a single NVIDIA H200 reports roughly 5–6 FPS at 1080p for SAM 3, versus 30+ FPS for SAM 2 on the same clip. Per Meta's paper, a single GPU sustains near-real-time for about 5 concurrent objects; you scale by adding GPUs (roughly up to 10 objects on 2×H200, 28 on 4×H200, 64 on 8×H200).
• SAM 3.1 helps the multi-object case via multiplexing, but a single object on one GPU is still slower than SAM 2. If SAM 2's single-object tracking already covers your need, you may not need to upgrade.

Troubleshooting

• 401/403 downloading the checkpoint — you haven't been granted access, or the CLI isn't logged in. Confirm approval on the model page and re-run hf auth login.
• CUDA version mismatch / "no kernel image" — your PyTorch CUDA build doesn't match your driver. Reinstall the cu128 wheel and verify with python -c "import torch; print(torch.cuda.is_available())".
• Out-of-memory on video — reduce resolution, track fewer concurrent objects, or shorten the clip; memory scales with both frame count and object count.
• Crash on Apple Silicon (MPS) — expected today; fall back to CPU via Ultralytics or move to a CUDA host.
• flash-attn build fails — it's optional. Skip the FlashAttention step; SAM 3 runs (slower) without it.
• Slower than you hoped — call set_image() once and reuse the state across prompts; re-encoding per prompt is the most common self-inflicted slowdown.

FAQ

Is SAM 3 free to use?

The code and checkpoints are released by Meta for research and application use, but the weights are access-gated on Hugging Face — you request access and agree to the license before downloading. Review the license terms on the model page for your specific use case.

What's the difference between SAM 3 and SAM 2?

SAM 2 masks one object per visual prompt (point or box). SAM 3 adds open-vocabulary text and exemplar prompts and returns every instance of a concept, with built-in detection. SAM 3 is larger and slower per frame, so SAM 2 is still the better pick for lightweight single-object interactive masking.

Do I really need an NVIDIA GPU?

For the official repo and any serious workload, yes. CPU works only through the Ultralytics wrapper and is far too slow for video. Apple Silicon (MPS) support is an open request, not a shipped feature, as of mid-2026.

Should I download facebook/sam3 or facebook/sam3.1?

Use facebook/sam3.1 — it's a drop-in update with faster multi-object video tracking and better re-identification. Same API, better numbers.

Can SAM 3 segment objects it has never seen a label for?

That's the point of open-vocabulary segmentation: you describe the concept in plain language ("cracked roof tile," "person wearing a helmet") rather than choosing from a fixed class list, and SAM 3 attempts to find every instance.

How much VRAM do I need for video?

Budget roughly 18–25 GB for a short single-object clip, and expect it to climb with clip length and object count. Multi-object, longer, or higher-resolution video can exceed 40 GB, which is why teams parallelize across GPUs.