North American Sinea ID and the Sorcerer's Apprentice problem

[upupa-epops]

Following up on this as we have a more detailed description of the taxon/observation inclusion criteria in a comment from loarie here:

New computer vision model with over 2,500 new taxa

loarie:
The logic for which observations are included is complicated. If you’re ok with a high level summary you can find one here: https://help.inaturalist.org/en/support/solutions/articles/151000170368
If you want more details, looking at the code is probably the best way to precisely explain it:
https://raw.githubusercontent.com/inaturalist/iNaturalistAPI/refs/heads/main/lib/vision_data_exporter.js
and here’s my attempt to do that in english looking at the code, but it likely doesn’t exactly match the logic in the code

Step 1 - sort out the taxonomy by considering

• Not globally extinct (no taxa marked as extinct at the global level).
• At or above species rank (no subspecies, varieties, or anything below species).
• Not hybrids (rank = hybrid or genushybrid are excluded).
• Active taxa only (inactive taxa are skipped).
• Enough observations: a leaf taxon must have at least 50 observations with photos to qualify.

Step 2 - consider obs in the taxonomy by considering

• Must have photos (observation_photos_count > 0).
• Must not have unresolved flags (any open flag disqualifies it).
• Must pass all quality metrics other than “wild” (Captive/cultivated observations are allowed)
• But failing other metrics (e.g. “evidence of organism,” “location”) excludes the record.
• Location constraints (for use in spatial datasets):
  • Must have valid coordinates (not missing or 0,0).
  • Must not be marked as “inaccurate location.”
  • Must have positional accuracy ≤ 50 km.
  • Must not have geoprivacy = private (uses true coordinates for geoprivacy = obscured)

Step 3 - consider photos based on

• No unresolved photo flags.
• Not hidden by a moderator.
• Up to 5 photos per observation can go into the training set (first one always used, up to 4 extras if room). One photo per observation can go into test and validation sets.
• No photo sharing across taxa: if a photo from an observation is in one taxon, its other photos can’t appear in another taxon.

Step 4 - each taxon must have at least:

• 50 training photos
• 25 validation photos
• 25 test photos
• Upper caps also exist (default: 1000 train, 100 val, 100 test per taxon).
• Spatial data capped at 100,000 points per taxon.
• Draw from observations with community_taxon first, for train top off with observations without community_taxon if needed