Yeah, I’m concerned about vision accuracy on avian hybrids & related species. It’s turning out to be a hard problem for the vision system. We don’t train on subspecies, and it may be that we shouldn’t train on any infraspecific taxa. I’m planning to do experiments next month to decide whether we should exclude avian hybrids in future models, or otherwise treat them differently.
The background is that previous versions of the model had far fewer avian hybrid photos to train on. We had a large growth in the number of avian hybrid identifications in the past year and for the first time, our (capped) training data had as many Mallard x American Black Duck photos as Mallard photos and American Black Duck photos. We didn’t exclude them from the training dataset because it wasn’t a known problem, but obviously I’m re-evaluating that now.