Insight in disagreed annotations

General
1

Hi all,

Just wondering is it possible to see where someone did disagree with an annotation I added (for example life stage)? This to learn and/or correct errors. Any tips or advice welcome.

1 Like
2

you have to get the details of each observation that has annotations via the API and parse through the annotation “votes” to see if there are disagreements. right now, it doesn’t look like any of your observations with life stage annotations have any disagreeing votes.

3

Because annotations can be disagreed with but not corrected by anyone except the observer/annotator and they don’t generate notifications, I always comment/tag the annotator if I come across wrong annotations.

13 Likes
4

Ah, I didn’t know that. That sounds like a good practice, thanks.

5

Would you care to explain what “the API” is in this context?

6

API = application programming interface.

information about iNat’s API can be found at https://api.inaturalist.org.

i used one of my Jupyter notebooks running Python to get information from the API and come to the conclusions in my previous post, but you can also write your own code / scripts to get information from the API.

1 Like
7

Thanks for checking pisum.
It’s a pity this is not default available in inaturalist.
Are you able to share your code in some way?

8

there’s not a straightforward way to look for the particular information you want because it’s a complicated request, and there are a lot of potential variations in how exactly any particular persion might want to filter for observations in their particular case. maybe one person wants any disagreement on a particular annotation. (that’s the way i did it.) but maybe another person wants only disagreements by a particular person on any annotation. and maybe another person wants another person wants only disagreements on any annotation by a particular person. each of these variations requires very different ways of parsing the data to get the desired result.

see the link to my Jupyter Notebook in my previous post. as it is currently saved, you would:

  1. uncomment (remove the leading #) from items / lines in the parse_fields definition for the async def get_obs which begin with label: 'annot_score_ls...'. (this will tell the code to return the overall scores for life stage annotations, which i’ve defined as 1 for the annotation, plus 1 for each agreement, minus 1 for each disagreement)
  2. set req_params_string = 'user_id=rudolphous&term_id=1'. (this tells the code to get your observations that have a life stage annotation.)
  3. set obs = await get_obs(req_params, get_all_pages=True, use_authorization=False). (the get_all_pages=True parameter will get up to the first 10,000 records for the parameters you defined in #2, which will be a high enough limit in your case.)
  4. starting at the top, run each cell that has code, through to the cell that includes obs = await get_obs.
  5. run the cells in the section that will write a CSV.
  6. look in the pane on the left, and you’ll find the resulting CSV. you can browse it directly in the browser or download it to view it using tools / applications on your own machine. look for any observations / records where any of the socres <1 in the life stage annotation score columns. these would be observations that have disagreements.

note that although score <1 is not a perfect indicator of disagreements, it should work in most cases and was the easiest way to find these without doing new coding. you could make it better by filtering the votes on the individual annotations for just disagreements and counting those vote records. any vote count > 0 would indicate there was a disagreement.

instead of looking yourself for scores <1 in step #6, you could also tell the code to filter for records that have scores <1 directly by adding a parameter in step #3 that would be something like post_parse_filter_function=(lambda x: x['annot_score_ls_adult'] < 1 or x['annot_score_ls_juvenile'] < 1 or ...), where you would define which particular life stage annotations you want to look for overall scores less than 1.

1 Like
9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.