Short version: Make the AI learn from its own mistakes by implementing into the algorithm a method to count disagreeing identifications and thus detect problematic taxa (i.e. those with a high amount of erroneous AI suggestions).
Long read:
Computer Vision works great for some taxa and areas and less so for other organisms and regions.
While I am strongly in favor of improving the relevance of locality, probably resulting in higher accuracy and/or more conservative suggestions (genus level and higher) (discussed in this topic), here I want to focus on another important aspect of AI errors:
Species that rarely can be IDed by photos to species level.
For many arthropod groups, there is that one species, that the AI loves to suggest, despite the fact that
a) the specimen clearly belongs even to a different genus, family...
and/or b) it is not at all occurring in that region
or c) it might be that species, but is impossible to exactly ID on photos
This could lead to a ‘vicious circle’: false IDs lead to more suggestions of that species, which lead to more false IDs by others.
Suggestions have been made to take curational steps and ‘block’ some species from the AI pool, if experts state that a correct ID is too difficult or unlikely to be made in the field. I am not in favor of this but rather want to ask:
Why not use the power of crowdsourcing to find those tricky taxa and implement this in the learning algorithm?
How could this be done? By looking at the ratio of disagreements for a certain taxon.
Example: Flesh flies (Sarcophagidae) are often IDed as Sarcophaga carnaria, although a species ID is almost impossible with in vivo photos. If those IDs were made using the AI suggestions and subsequently get corrected by other users to genus level, then the algorithm could learn to avoid suggesting a species in these cases and become more conservative. One just has to define a threshold for the disagreement ratio for a certain taxon.
I would be interested to learn if this would be feasible to implement into the system, and also if there would be any downsides of such a system.
One could imagine that this might negatively affect the AI accuracy of species that are only in parts of their range difficult to ID, as in other areas there would be no similar species around and the ID thus straightforward. However, improving a correct genus to species level is easier then correcting an incorrect species choice.
And ideally, to go one step further, location plus the ‘disagreement ratio’ could be combined, such as e.g. in Western Europe the AI might suggest Erinaceus europaeus for a hedgehog, but in Russia, where several species occur, it would prioritize the genus (similar case with American Crow on the pacific coast vs. possible confusion with Fish Crow in the east).
Let me hear what you think.
