Run llgx16vjerror
3 Sept 2026, 11:57 pm · 2 frames × 6 model arms = 12 cells. Every frame was tiled once and the identical tile set was sent to every arm.
- rf-detr on cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live
- sam3 on cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live
- countgd on cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live
- grounding-dino on cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live
- owlv2 on cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live
- gemini-flash on cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live
Configuration
Chosen once, applied to every arm. This is the fairness contract the whole comparison rests on.
- Tile size
- 512 px
- Overlap
- 20%
- Merge (IoS)
- 0.3
- Green prefilter
- on
- RF-DETR Small/Medium (+SAHI)SIMULATED PROFILEApache-2.0
- SAM 3.1 (exemplar + concept)SIMULATED PROFILESAM License (commercial use permitted, carve-outs + patent clause)
- CountGDSIMULATED PROFILEMIT
- Grounding DINO (open Swin-B)SIMULATED PROFILEApache-2.0
- OWLv2 (image-conditioned one-shot)SIMULATED PROFILEApache-2.0
- Gemini Flash (tiled)SIMULATED PROFILEClosed commercial API
The Excess Green prefilter skipped 72 of 72 tile-visits. A skipped tile is scored as containing nothing, so any plant inside one counts as a miss — recall is reported against the whole image, never against the tiles that were sent.
Totals
Results grid
Counts, not scores. A count next to its truth is the number Habitat NZ's consent condition actually asks for; accuracy is the Evidence page's job.
| Frame | Truth | RF-DETR Small/Medium (+SAHI) SIMULATED PROFILE | SAM 3.1 (exemplar + concept) SIMULATED PROFILE | CountGD SIMULATED PROFILE | Grounding DINO (open Swin-B) SIMULATED PROFILE | OWLv2 (image-conditioned one-shot) SIMULATED PROFILE | Gemini Flash (tiled) SIMULATED PROFILE |
|---|---|---|---|---|---|---|---|
| CoFly-WeedDB — real UAV frame 1 (4.9 m AGL) 5.74 mm/px | none | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles |
| CoFly-WeedDB — real UAV frame 3 (4.9 m AGL) 5.74 mm/px | none | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles | —⚠ failed — · 0 ms 0/6 tiles |
The small figure beside each count is the signed count error against that row’s truth. It is kept signed on purpose: a model that misses half the plants and hallucinates the same number again has a perfect count and a useless map, and only the sign of the error tells you which of those you are looking at.
These are raw counts — every detection the arm returned, at no confidence threshold. They will run high, and they are not the figure the Evidence page reports: there each arm is scored at its own best-F1 operating point, because raw confidences are not comparable between providers. Use the slider in the inspector below to see what a threshold does to both the marks and the number.
Inspect a cell
Put the detections on the pixels. Toggle arms on and off, scroll to zoom to 1:1, and see whether the count is plants or schist. Layer visibility, pan and zoom are in the viewer itself.
CC BY 4.0 — CoFly-WeedDB (Krestenitis et al.), DJI Phantom 4 Pro over cotton field, Larissa, Greece — zenodo.org/records/6697343
Applies to the marks and the numbers together, so the table always describes what you can see.
This frame has no ground truth, so there is nothing to score against — the arms can be compared to each other and to your own eyes, and that is all. Precision and recall on an unlabelled frame would be invented numbers.
| Model | Access | Detections ≥ threshold | Tiles | |
|---|---|---|---|---|
| RF-DETR Small/Medium (+SAHI) | SIMULATED PROFILE | 0 | 0 / 6 | simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live |
| SAM 3.1 (exemplar + concept) | SIMULATED PROFILE | 0 | 0 / 6 | simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live |
Run manifest
Written when the run finished, and attached to every export. A count is worth what the trail behind it is worth.
- App version
- 0.1.0
- Run id
- llgx16vj
- Created
- 3 Sept 2026, 11:57 pm
greedy nearest-centroid within max(3px, 1.5 × median ground-truth box edge); highest-confidence first, one detection per plant; point annotations counted as 8px across; planted hard negatives are unmatchable, so a detection on one scores as a false positive (centre distance, never IoU: at 5–15 px per plant a one-pixel shift moves IoU more than the gap between two models)
- SIMULATED ARMS IN THIS RUN (6 of 6): RF-DETR Small/Medium (+SAHI) (rf-detr), SAM 3.1 (exemplar + concept) (sam3), CountGD (countgd), Grounding DINO (open Swin-B) (grounding-dino), OWLv2 (image-conditioned one-shot) (owlv2), Gemini Flash (tiled) (gemini-flash). Their numbers come from a simulated profile, not from live inference.
- SIMULATED means no inference ran on this imagery. A simulated arm draws detections from a deterministic, seeded error model whose recall/false-positive/jitter parameters are read off published benchmarks for that model (see docs/RESEARCH.md), applied to the known ground truth. It is a PRIOR describing what that model would plausibly do, not a measurement of what it did, and it must never be quoted as this model's accuracy on Central Otago imagery. Every simulated figure is replaced by a real one the moment that arm runs live.
- IMAGE LICENCES IN THIS RUN: CC BY 4.0 (2 images).
- Attribution travels with the pixels: every CSV row and GeoJSON feature carries the licence and attribution of its source image, so a count lifted out of this app into a spreadsheet keeps its credit. CC BY requires that credit wherever the derived count is published; CC BY-SA additionally share-alikes anything derived from it.
- Georeference quality: pixel-space (demo) — approximate; real flights get EXIF/RTK georeferencing. Export coordinates are image pixels (x = column, y = row, top-left origin) in original full-resolution space. Nothing here is a map projection.
- No ground truth for 2 image(s): cofly-01, cofly-03. Counts there are model-vs-model only; precision and recall are undefined without human truth.
- Excess Green prefilter skipped 72 of 72 tiles. A skipped tile is scored as containing no detections, so any plant inside one counts as a miss — recall is reported against the whole image, never against the tiles that were sent.
- 12 cell(s) failed: rf-detr/cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; sam3/cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; countgd/cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; grounding-dino/cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; owlv2/cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; gemini-flash/cofly-01: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; rf-detr/cofly-03: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; sam3/cofly-03: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; countgd/cofly-03: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; grounding-dino/cofly-03: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; owlv2/cofly-03: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live; gemini-flash/cofly-03: simulated profile needs known ground truth — this frame has none; annotate it (human truth still won't feed the simulation) or add the provider's API key to run this lane live. A failed cell contributes no detections; treat its count as absent, not as zero.