Label aggregation

General
1

Hello,
There’s a lot of talk about ResearchGrade and taxa identification.
i was just wondering one thing:

iNaturalist released several datasets for us to work (in competitions each year for example), and the images are associated to a ground truth label for us to do classification based problem. But how exactly are these “ground truths” obtained.

I see a lot the work “consensus” used on the website and in the forum. And each consensus can be broken in time if enough people disagree. But do we all have the same weight on the decision or is there some internal scoring system explained somewhere?

I find this system very useful as this is a large scale experiment with so much data and annotations ! And for now I don’t known if the final label choice is using majority vote consensus or something potentially more complex. Just wondering the inner workings of this beautiful system I guess

1 Like
2

No user gets more weight than any other user. One identifier, one vote.

When there are disagreeing IDs, there must be a more than 2/3rds majority in agreement for the observation to reach Research Grade.

See What does the Community Taxon Mean?

3 Likes
3

You can see the info in the above link on any observation along with a graphic for how the CID is determined by clicking on “What’s This?” next to “Community Taxon”

1 Like
4

Thank you very much for this clear explanation !

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.