Evidence — accuracy vs mm-per-pixel
The chart a count has to stand on: how each model's recall collapses as plants shrink below ~5 pixels.
Recall against ground sample distance
One line per model. Each point is mean recall over the scored frames in that mm-per-pixel bucket, at that model's own best-F1 threshold. Ground sample distance — not model choice — sets the ceiling.
DJI Mavic 3E — wide equivalents: 1 mm/px ≈ 3.8 m · 2 mm/px ≈ 7.5 m · 8 mm/px ≈ 30 m. Nominal optics — budget usable pixels at roughly nominal ÷ 1.5 once lens MTF, demosaicing, JPEG and motion blur are counted.
- SAM 3.1 (exemplar + concept)SIMULATED PROFILE10 scored
- RF-DETR Small/Medium (+SAHI)SIMULATED PROFILE10 scored
- GeCo2SIMULATED PROFILE10 scored
- YOLO26-s/m + P2 head (1280–1536, SAHI)SIMULATED PROFILE10 scored
- T-Rex2 (interactive visual prompt)SIMULATED PROFILE10 scored
- CountGDSIMULATED PROFILE10 scored
- OWLv2 (image-conditioned one-shot)SIMULATED PROFILE10 scored
- Gemini Flash (tiled)SIMULATED PROFILE10 scored
Leaderboard
Sorted by mean F1. Every arm is judged at its own best-F1 operating point, because raw confidence scores are not comparable between providers — anything else would score the calibration, not the detector.
| # | Model | Status | F1 | Precision | Recall | Count MAE | Count bias | Cells |
|---|---|---|---|---|---|---|---|---|
| 1 | SAM 3.1 (exemplar + concept) sam3 | SIMULATED PROFILE Simulated — ROBOFLOW_API_KEY is not set. | 65.1% | 81.5% | 56.7% | 131.5 | -131.1 | 10 |
| 2 | RF-DETR Small/Medium (+SAHI) rf-detr | SIMULATED PROFILE Simulated — ROBOFLOW_API_KEY is not set. | 64.1% | 84.2% | 54.6% | 143.1 | -142.9 | 10 |
| 3 | GeCo2 geco2 | SIMULATED PROFILE Simulated — local weights only (GitHub jerpelhan/GECO2) — no hosted API in the four-key plan. | 61.6% | 68.7% | 57.9% | 98.3 | -94.5 | 10 |
| 4 | YOLO26-s/m + P2 head (1280–1536, SAHI) yolo26-p2 | SIMULATED PROFILE Simulated — hosted on Replicate but not wired here; AGPL must be resolved before delivery. | 60.6% | 80.3% | 52.0% | 138.7 | -137.3 | 10 |
| 5 | T-Rex2 (interactive visual prompt) t-rex2 | SIMULATED PROFILE Simulated — DeepDataSpace account not opened (same vendor as DINO-X, decision G3). | 59.0% | 70.4% | 52.6% | 118.7 | -118.1 | 10 |
| 6 | CountGD countgd | SIMULATED PROFILE Simulated — Hugging Face Space / local build — no hosted API in the four-key plan. | 54.1% | 59.6% | 53.4% | 104.0 | -93.2 | 10 |
| 7 | OWLv2 (image-conditioned one-shot) owlv2 | SIMULATED PROFILE Simulated — not on Roboflow serverless; Replicate route not wired here. | 32.3% | 30.2% | 39.3% | 128.3 | -43.7 | 10 |
| 8 | Gemini Flash (tiled) gemini-flash | SIMULATED PROFILE Simulated — OPENROUTER_API_KEY is not set. | 19.8% | 18.1% | 25.9% | 141.9 | -13.7 | 10 |
Bias matters more than MAE: a consistently −20% model can be calibrated; a random one cannot. MAE is the mean absolute miss in plants per frame; bias is the same error kept signed, so a negative number is systematic under-counting and a number near zero with a large MAE means the misses cancel by luck rather than by skill.
Where each model sits
Two models can share an F1 and behave completely differently on a consent job. This separates the cautious arms from the enthusiastic ones.
Each point is that model at its own best-F1 threshold. Bottom-right is a keen model that over-reports; top-left is a cautious one that misses plants. Only the top-right corner supports a bare number in front of a panel.
- SAM 3.1 (exemplar + concept)SIMULATED PROFILE
- RF-DETR Small/Medium (+SAHI)SIMULATED PROFILE
- GeCo2SIMULATED PROFILE
- YOLO26-s/m + P2 head (1280–1536, SAHI)SIMULATED PROFILE
- T-Rex2 (interactive visual prompt)SIMULATED PROFILE
- CountGDSIMULATED PROFILE
- OWLv2 (image-conditioned one-shot)SIMULATED PROFILE
- Gemini Flash (tiled)SIMULATED PROFILE
Reading it honestly
What these numbers are, and what they are not. Read this before quoting anything above.
- Simulated arms. SAM 3.1 (exemplar + concept), RF-DETR Small/Medium (+SAHI), GeCo2, YOLO26-s/m + P2 head (1280–1536, SAHI), T-Rex2 (interactive visual prompt), CountGD, OWLv2 (image-conditioned one-shot), Gemini Flash (tiled) ran a deterministic error profile derived from published benchmarks, not the model. Those lines demonstrate the harness; they are not measurements, and each one is replaced by a real number the moment that arm runs live with its key in place.
- How a detection becomes a hit. greedy nearest-centroid within max(3px, 1.5 × median ground-truth box edge); highest-confidence first, one detection per plant; point annotations counted as 8px across; planted hard negatives are unmatchable, so a detection on one scores as a false positive (centre distance, never IoU: at 5–15 px per plant a one-pixel shift moves IoU more than the gap between two models). At 5–15 px per plant IoU is noise — a one-pixel shift on a 7 px rosette moves it more than the gap between two models — so the harness never scores mAP@0.5:0.95 here. The radius comes from the ground truth itself, identically for every arm, so no model can be flattered by a looser tolerance than its rivals.
- Spatial validation. Random-split numbers flatter: tiles from the same flight share lighting, phenology and substrate, so a model can memorise the site rather than the plant. Whole-flight holdouts are the honest test, and a 2025 grassland-flower study saw recall collapse to 0.7–27.6% off-site. The harness runs both splits when real flights land, and reports them side by side.
- Two regimes, two questions. Scattered plants can be counted as individuals. Dense mats cannot: every box detector saturates, and the defensible measure becomes cover fraction or density per m², not a count of individuals. A leaderboard row averaged over both regimes hides that — check the per-frame results on the run page before quoting one number for a whole site.
- What is missing from the averages. 0 of 80 cells could not be scored at all: real UAV frames carry no ground truth until someone annotates them, so they produce a count with no accuracy claim beside it. Hard negatives (schist chips, lichen, droppings) stay in every frame and are unmatchable by design: a detection that lands on one is a false positive, which is exactly the failure mode worth measuring.