When I see an unfamiliar identifier contribute to one of my observations, I often click through to their profile and other identifications to try to get a sense of their expertise.
To that end, I updated my Chrome extension to surface the sum of leading and improving identifications the user has made of the identified taxon, as well as the overall breakdown on mouseover:
Obviously, this is a crude heuristic–I’m sure there are expert identifiers regularly joining the site, who don’t yet have a track record–so this isn’t meant to pass definitive judgment on anyone, just serve as a starting point for getting to know other users, an ad hoc reputation score.
Thoughts? Do you find that sum useful, or would you rather see the data represented (and/or presented) in a different manner?
It would be interesting if it could link to a user’s Maverick page so you can quickly access those IDs in context and see their discussions on those observations. Maverick number is especially divided since a user could be either accurate or inaccurate to best ID in those cases, only reviewing context can familiarize you with the user’s knowledge re: Maverick category.
I like it! A potential improvement would be to display it as a ratio instead as the number of (leading + improving) / supporting. e.g. 20 / 40 since the number of supporting IDs does have some info in it.
Also very cool how it’s in the identify modal too
Could also be cool to display the stats for yourself as well
If we’re looking for ideas, maybe having the tool list out the top (or top two) phyla or classes that a person has identified in would help contextualize their focus as an identifier. I personally don’t find leading / improving / supporting stats all that informative.
Well, I figured people would already have a sense of how well they themselves know a particular taxon :) But that would be easy to add.
I think that would be interesting to do as a tooltip over the user’s name or profile picture (or on the profile itself), but is not exactly the same thing I was trying to do here, which was contextualize a specific identification: “Hey, I don’t recognize this person–do they meaningfully ID this species/family/whatever a lot?”
Both other categories are definitely useful data, but I think maybe too context-dependent to easily and meaningfully display in a simple manner. I wanted to display a single, easily digestible number, and I thought leading + improving was the best way to get a sense of how often someone is really driving a particular ID. There is the additional breakdown in the tooltip if desired, and of course the native /identify page for the full data. But I’m interested to see if there’s a large group of people who feel differently than me, keeping those same goals in mind.
It does have a feature toggle, so it can be disabled while preserving the rest of the extension’s functionality. It would also be possible to disable it on just the /identify page, but as @kevinfaccenda said, I actually like having it there most of all.
‘I was first’ across 24/7 and geography doesn’t necessarily mean anything.
Since we can only measure quantity (not quality) and activity, perhaps the popup doesn’t serve a useful purpose. One ID versus 300 or 3K - that I can use. I pick that up at first glance. Saves me clicking thru to the leaderboard or profile sometimes.
i guess i was more concerned that regular folks might not understand that the implications of using the feature, without some sort of warning. it might not be obvious to folks that weirdness in the Identify page could be linked to the use of the feature.
seems like you would need at least one extra query per id taxon just to show the summary number, and 4 per id taxon to show the breakdown by category. so that can add up fast.
oh, this is neat! I’ve been wanting to work on a similar project. the way i was thinking about it was: given all the times the identifiers has identified this taxon, how often has their ID been accepted vs overruled?
i haven’t gotten very far – i’m a rudimentary programmer, limited to matlab and python, and I’m still wrapping my head around the API. but what I like about this approach is that it could be a step toward transforming the community standard from 2/3 support to 95% confidence. it could also help produce a more informative compare function by leveraging the confusion matrix.
I do like stats and numbers, as they motivate me. So I do like to see this kind of stats for my own stuff for sure.
However, I don’t really see the usefullness of those numbers for what you want to determine. We quite recently had a discussion about that in the forums as well, I don’t remember under which topic, though.
It depends so much on what you ID (e.g I would imagine that it is harder for bird IDers to have a high leading score, as there are sooo many identifiers and one needs to be really quick to be the first), how much you ID (someone doing 10.000 IDs a month will have different scores then someone doing 500 a month) and where and when you ID (IDer density).
I think in the other discussion we had, some people mentioned that people mainly doing unknows can have really high leading scores easily, pushing observations from unknowns simply to kingdom, while experts IDing maybe fungi comb through hundreds of observations they might not be able to do much more for then agreeing - simply because distinguishing features are not displayed.
So, I don’t find those numbers very useful for judging IDer competence at all, actually.
It is much more helpful to just “get to know” their work through experience, I don’t think there are easy shortcuts.
That is even worse for determining something, I think. What info does this ratio actually hold? Is someone doing 1.000 improving IDs and 1.000 supporting IDs (1:1) really the “better IDer” then someone doing 1.000 improving IDs and 9.000 supporting in a given timeframe (1:9)?
I didn’t mean to reduce the ratio, but to keep it as two whole numbers. That way if somebody has say 2 leading IDs, but 20 supporting IDs, it means that they are more familiar with the taxon than 2 leading IDs and 0 supporting IDs.
The number and “suggested an ID” is added after the whole string, not just username, so it looks like “user suggested an ID (46) suggested an ID”.
PS: May I ask you for some more features of the extention?
I often need to view observations for specie of the observation, my observation of this specie and all observations of the observer for the day of current observation.
Say it is a spider obs. If you are already there = trusted identifier.
But, if you haven’t seen it, and I have a row of unfamiliar names - this is a shortcut showing me who I could try first for a ‘please’ @ mention. For familiar names it puts a numerical value on trusted identifier versus new to me name with only one or two IDs of that. If they are a new taxon specialist … I will see that number racking up as they settle in to using iNat. I remember when your name didn’t click spider in my mind.
Only 257 of us use this app (from Chrome?) I rely on the visual guide to how confident CV is. Then the copy coordinates. Much appreciated. Regular folk are missing out.
