Description of need:
Related to @conboy 's post at https://forum.inaturalist.org/t/community-annotation-challenges/5488, we have a community that wants to improve iNat data through annotation. One motivating facet is to see how we’re performing in relation to our peers, so just as we have fast methods of checking where we stand in terms of number of observations, species, and identifications, could we please have an efficient way to show our standings in terms of number of annotations?
Feature request details:
A conventional leaderboard, and/or similar to
a new API endpoint /v1/observations/annotators
@benarmstrong reviewed all of the other endpoints and ruled them out because:
the ones returning user stats returned only top observers (by # of obs or # of species) or top identifiers, but not the users making the annotations
even when an annotated_observation_count was included, it wasn’t related to the observation query, but just to the user’s overall stats
the only recourse would seem to be to exhaustively download all observations of interest, then tally up all annotations made on them by user making the annotation, which is incredibly inefficient
if you can constrain your “peers” to a specific list of users, then you could always just get an annotated observations count for each of those users in the list (ex. https://api.inaturalist.org/v1/observations?per_page=0&annotation_user_id=pisum). note that this will count only observations, not annotations, and it’s based on original annotations, not also agreements.
“Peers” is a very open-ended group, dependent on the slice of data we’re looking at. It may not even be people within our Discord community, or people we know from iNat outside of it. It would be nice to know all the people who also cared about the part of the iNat data we’re working on and sharpen each other by comparing our stats, but we don’t really know that in advance. Does that make it clearer?
In a Discord context we’d use this API endpoint to create displays from observation queries on-the-fly (e.g. a query that will take N seconds for N people or crunch even more than a few 200-observation pages is just not going to cut it …) much in the same way as we do with identifiers with /v1/observations/identifiers. Let’s say our interest is Hempitera in Northeastern US & Canada, we can say:
and produce a similar display for annotators. It should list not the overall annotation stats for those users, but the # of annotations made by annotators for the matching data set.
The web equivalent of the identifiers leaderboard is linked from the title of the display in the screenshot above:
We understand that this may not be something that is given much priority for the web, but it would be nice to have a view=annotators as well. At least having the API endpoint would allow us to fill in the gaps for these needs even if it is outside any immediate plans for the web.
that’s fine. my thought was that you could have a better-than-nothing way of fostering competition strictly within an interested group, but if that’s not a path worth considering, then i guess you’ll just have to wait for a system change, unless you code and submit the changes yourself.
Understood. While it’s not entirely out of the question to make the change myself, I’m hoping we’ll get to the point where there’s more community demand for this beyond our relatively small segment of the wider iNat community.
While we wait to see if this feature request gets implemented, here’s my informal list of the top annotators, based on a survey of the top 500 identifiers plus other profiles I’ve encountered. If you’re on this list, thanks for your efforts to improve phenology data on iNaturalist! If you have over 100K annotations and you’re not on this list, please let me know so I can add you!
Thanks, I’ve updated my personal list. But unfortunately it’s not a matter of forgetting anyone, it’s a matter of not being able to find them in the first place. Please vote for the feature request!
i took a random sample of 10000 observations with annotations (based on verifiable=any&term_id=17,33,1,22,36,9,12&order_by=random). taking the top 25 users with annotations in that set and getting their total annotated observation counts in the system, i’m fairly certain that your original top 13 + calvertm + lwnrngr represent the true top 15 annotators in the system at the moment.
UPDATE:
just to give you at least as many users as in your original list of 35, i repeated the same sort of process, but i ran the top 77 annotators from my set of 10000 random observations, and it looks like that gives me a fairly good idea of the top 39 or so. my guess is that it would would be possible to get a good idea of the top 50 by running 150 users, and 100 might take 1000 users. beyond that, i’d probably have to start with a larger random observation sample size, which would be a little more complicated to do (but not too hard).
