Run lujssp9mdone
3 Sept 2026, 11:47 pm · 3 frames × 1 model arm = 3 cells. Every frame was tiled once and the identical tile set was sent to every arm.
Configuration
Chosen once, applied to every arm. This is the fairness contract the whole comparison rests on.
- Tile size
- 512 px
- Overlap
- 20%
- Merge (IoS)
- 0.3
- Green prefilter
- on
- SAM 3.1 (exemplar + concept)SIMULATED PROFILESAM License (commercial use permitted, carve-outs + patent clause)
The Excess Green prefilter skipped 16 of 33 tile-visits. A skipped tile is scored as containing nothing, so any plant inside one counts as a miss — recall is reported against the whole image, never against the tiles that were sent.
Totals
Results grid
Counts, not scores. A count next to its truth is the number Habitat NZ's consent condition actually asks for; accuracy is the Evidence page's job.
| Frame | Truth | SAM 3.1 (exemplar + concept) SIMULATED PROFILE |
|---|---|---|
| Hero scene — 1.0 mm/px — scattered with hard negatives 1 mm/px | 80 | 155+75 — · 1.2 s 7/15 tiles |
| Synthetic ladder — 1.0 mm/px — scattered 1 mm/px | 49 | 102+53 — · 1.3 s 5/9 tiles |
| Synthetic ladder — 4.0 mm/px — scattered 4 mm/px | 27 | 80+53 — · 1.0 s 5/9 tiles |
The small figure beside each count is the signed count error against that row’s truth. It is kept signed on purpose: a model that misses half the plants and hallucinates the same number again has a perfect count and a useless map, and only the sign of the error tells you which of those you are looking at.
These are raw counts — every detection the arm returned, at no confidence threshold. They will run high, and they are not the figure the Evidence page reports: there each arm is scored at its own best-F1 operating point, because raw confidences are not comparable between providers. Use the slider in the inspector below to see what a threshold does to both the marks and the number.
Inspect a cell
Put the detections on the pixels. Toggle arms on and off, scroll to zoom to 1:1, and see whether the count is plants or schist. Layer visibility, pan and zoom are in the viewer itself.
CC0 (generated) — Synthetic scene — EmerTech generator
Applies to the marks and the numbers together, so the table always describes what you can see.
| Model | Access | Count | TP | FP | FN | Precision | Recall | F1 | Count error |
|---|---|---|---|---|---|---|---|---|---|
| SAM 3.1 (exemplar + concept) | SIMULATED PROFILE | 102 | 31 | 71 | 18 | 30.4% | 63.3% | 41.1% | +53 |
Scored against 49 plants and 39 planted hard negatives by greedy centre-distance matching, highest score first, one detection per plant, within 16.9 px (max 3 px, 1.5 × the median ground-truth edge). Never IoU: at 5–15 px per plant a one-pixel shift moves IoU more than the gap between two models. A detection landing on a planted negative is a false positive — that is what the decoys are for.
These figures are at the threshold on the slider (0.00). The leaderboard reports each arm at its own best-F1 operating point instead, because raw confidences are not comparable across providers — so the two will differ, and both are stated rather than one quietly replacing the other.
Run manifest
Written when the run finished, and attached to every export. A count is worth what the trail behind it is worth.
- App version
- 0.1.0
- Run id
- lujssp9m
- Created
- 3 Sept 2026, 11:47 pm
greedy nearest-centroid within max(3px, 1.5 × median ground-truth box edge); highest-confidence first, one detection per plant; point annotations counted as 8px across; planted hard negatives are unmatchable, so a detection on one scores as a false positive (centre distance, never IoU: at 5–15 px per plant a one-pixel shift moves IoU more than the gap between two models)
- SIMULATED ARMS IN THIS RUN (1 of 1): SAM 3.1 (exemplar + concept) (sam3). Their numbers come from a simulated profile, not from live inference.
- SIMULATED means no inference ran on this imagery. A simulated arm draws detections from a deterministic, seeded error model whose recall/false-positive/jitter parameters are read off published benchmarks for that model (see docs/RESEARCH.md), applied to the known ground truth. It is a PRIOR describing what that model would plausibly do, not a measurement of what it did, and it must never be quoted as this model's accuracy on Central Otago imagery. Every simulated figure is replaced by a real one the moment that arm runs live.
- IMAGE LICENCES IN THIS RUN: CC0 (generated) (3 images).
- Attribution travels with the pixels: every CSV row and GeoJSON feature carries the licence and attribution of its source image, so a count lifted out of this app into a spreadsheet keeps its credit. CC BY requires that credit wherever the derived count is published; CC BY-SA additionally share-alikes anything derived from it.
- Georeference quality: pixel-space (demo) — approximate; real flights get EXIF/RTK georeferencing. Export coordinates are image pixels (x = column, y = row, top-left origin) in original full-resolution space. Nothing here is a map projection.
- Excess Green prefilter skipped 16 of 33 tiles. A skipped tile is scored as containing no detections, so any plant inside one counts as a miss — recall is reported against the whole image, never against the tiles that were sent.