The current approach for annotations (Adult, Alive, Flowering, etc.) is that any user can apply an annotation and other users can then vote that up or down, but subsequent annotation choices are limited by what other users have already chosen. I’m proposing a system that more closely matches the way we reach consensus on IDs. I believe it would improve the accuracy of annotations on iNat and be easier for users to understand.
Let me outline the issue: Let’s say I come across another user’s observation that is actually of a juvenile male bird, maybe a Ruby-throated Hummingbird. Maybe, being over-confident about my knowledge of the species, I’m sure it’s an adult female and I choose the “Adult” and “Female” annotations. Any other iNat user who then views that observation cannot contribute their opinion that it’s really a juvenile male. All they can do is down-vote (or up-vote) the “Adult” and “Female” annotations. That’s because for most annotations selecting one option causes the others to disappear.
This appears to be happening because reasonable limitations on one person’s actions are mistakenly being applied across all iNat users. It’s good that iNat won’t allow me to say that a single bird is both “Adult” and “Juvenile”, but it’s not good that my identifying it as an adult prevents your identifying it as a juvenile.
I accept that this is an infrequent issue, but given the vast numbers of iNat observations it’s one that does have some impacts:
Some observations will be “stuck” with incorrect annotations unless the user who originally applied them can be persuaded to remove them. Contrast this with IDs, where the community can overrule an initial bad ID.
If the user who applied the incorrect annotation leaves iNat, there is no way to fix the incorrect annotation. Again, that doesn’t happen with incorrect IDs, which can be outvoted by correct IDs even if the first identifier is AWOL.
The value of iNat’s data for research use is diminished either because annotations are known to include unfixable annotation errors, or because observations need to be excluded where there’s any dispute over annotation accuracy.
I propose that iNat applies to annotations a slightly simplified version of the logic that is used for IDs. In the description below I used the term “consensus” to describe an annotation value that has sufficient support for in iNat to return it as a search result or supply it as part of an RG data feed.
- iNat modifies its schema to store annotation values tagged with the ID of the user applying them.
- In the iNat UI, conditional logic (e.g. “hide life stage choices if a life stage is already selected”) is applied on a user-specific basis.
- If there are no disputes, then a single “vote” is sufficient for an annotation value to be regarded as the consensus for an RG entry (i.e. one vote for “Adult” is sufficient for iNat to say the organism is an adult)
- If any value for a particular annotation is disputed, then >2/3 support is required for a value to have consensus support. (I say “Adult” and “Female”, you say “Juvenile” and “Male”. Neither annotation has consensus support. When two more users vote for “Female” that gains consensus support.)
- For incompatible annotations (“Adult” and “Juvenile”, “Flowering” and “No Evidence of Flowering”) there’s an edge case where there could be > 2/3 support for more than one option. The result should be no consensus for any option.
- Where multiple annotation values may be true simultaneously the above restriction should not apply (e.g. a plant can be both “Flowering” and “Fruiting” so long as there’s >2/3 support for those annotation values and <2/3 support for “No Evidence of Flowering”.)
Other logic may work better than my proposal above. The core point is to apply a consensus process to annotations that’s similar to what we have for IDs.
This proposal came out of a discussion here about the new “No Evidence of Flowering” annotation value.