The accuracy rates for the vision model and for RG observations are not comparable. When assessing the RG obs we were comparing that to expert IDs (assuming the expert ID is “the truth”), but when assessing the vision model we’re comparing it to RG obs (assuming the RG obs is “the truth”). Machines might be better than the iNat community at identifying insects, but we haven’t shown that.
Sometimes. Depends on the vision score and the nearby obs frequency.
We’re getting to the edge of my understanding here, so maybe @alex, @loarie, or @pleary can correct me, but the model outputs taxa and “scores”, where scores are values between 0 and 1, e.g taxon: 1234, score: 0.85. I’m told the score should not be considered a metric of “confidence” or “probability” and it should mainly be used for ordering outputs, but if that’s the case, why isn’t it an integer? I don’t know. Anyway, the fact that it’s not really a measure of confidence is why we don’t show it in any interface, b/c it’s way too tempting to think of it that way and rely too much on the cryptic opinion of this black box and not your own judgement.
To your question, the influence of the obs frequency is going to depend on the ancestor we’re using and how much more common a given taxon is within the search area, e.g. if we get a bunch of vision results, determine the ancestor is Canidae, and Coyote is WAY more common in the search area than Red Fox, then Coyote is going to get more of a boost than if we used Mammalia as the common ancestor, which might include Raccoons and other common things that would make Coyote less impactful relative to other mammals.