LocateAnything: 10x Faster Visual Grounding

🎯 NVIDIA released LocateAnything

A 3B vision-language model for visual grounding that runs at ~10× the throughput of comparable models — with no accuracy loss.

Give it an image (or video) + a natural-language query → it returns bounding boxes or coordinate points for the matching region.

The key idea → Parallel Box Decoding (PBD)

Most VLMs serialize a box into coordinate tokens and decode them one by one — slow, and it breaks the geometry (x1, y1, x2, y2 are structurally coupled). PBD predicts the whole box as a single atomic unit in one forward pass. Faster *and* more geometrically consistent.

⚡ Speed (single H100)

▸ 12.7 boxes/sec — LocateAnything ▸ 5.0 — Rex-Omni (2.5× slower) ▸ 1.1 — Qwen3-VL (10× slower) Runs Fast Mode (parallel) by default, with automatic fallback to Slow Mode (autoregressive) when an output looks unreliable — you keep the speed without losing robustness.

🎯 Accuracy

SOTA on LVIS, ScreenSpot-Pro, M6Doc & more. On LVIS it reaches 31.1 vs 20.7 for Rex-Omni at the strict IoU=0.95 threshold.

🌿 One model and five tasks

▸ Object detection ▸ Referring-expression grounding ▸ OCR localization ▸ Document-layout grounding ▸ GUI grounding

🏛 Architecture

MoonViT-SO-400M vision encoder + Qwen2.5-3B decoder. Part of NVIDIA's Eagle VLM family.

📊 Training data

12M images · 138M queries · 785M bounding boxes — across natural scenes, documents, GUIs, robotics & driving.

⚠️ The catch

Weights are on Hugging Face, but under a non-commercial research license. Great for research — not for production (except NVIDIA & affiliates).

Why it matters:

Throughput has been the real bottleneck for visual grounding in agentic "computer-use" and robotics pipelines. A 10× speedup at just 3B params makes near-real-time grounding on a single GPU realistic.