We have the goalposts for most IDs and most observations, which are great I think in inspiring action, but can also lead to misuse… as touched on in threads like these:
Why not make a new or connected goalpost for accuracy or another metric ?
I’m really not sure what this would or could look like in reality. ( interested to hear suggestions if there are any… ) I just think, fundamentally, shifting the goalpost slightly somehow, so its not just about quantity but also about quality… or adding an additional goalpost …could help remedy some of the above issues and connected problems with accuracy of iNat data and the CV, etc.
Personally though, I haven’t had too many problems that would be solved by “gamifying accuracy”. Accuracy problems usually occur in lesser known taxa, by people who think they are being accurate. They’re not usually caused by leaderboard chasers. On the flip side, the biggest issue I have with those people is the occasional 500+ notification spam when someone wakes up one day and decides they want to be #1 for a particular taxon.
I am in favour of a better way to rank identifiers though, if for no better reason than to help people work out who has the required knowledge. To that end though, I don’t care what their credentials are. So long as you can accurately identify an organism and want to be the #1 ranked identifier (by whatever metric), who cares if you’re a global authority on orcas, or if you’ve just watched Free Willy a few times?
Yeah. I think there is a lot that could be done in support of iNat’s objectives to gamify activities other than accumulating IDs. The leaderboard for species number is a beginning. The Badge system in iNat Forum is a model that could be adopted in some form. Metrics around accuracy have possibilities. I’m not up on learning theory but I’m guessing there’s a bunch of stuff that could be done to focus folks, particularly younger folks, on learning about biodiversity. Even just allowing people to click on a thank you box when they receive helpful comments with IDs would promote useful behaviour and give some kudos where deserved.
I really like this concept, although I worry that its implementation may not be feasible. In the last few months, I’ve been struggling to keep up with an overzealous identifier who has a tendency to agree with any identification suggestions presented to them. This results in a myriad of incorrectly identified research grade observations which slip past my radar and that of other identifiers, or, worse yet, erroneous community IDs. It got to the point where I rarely found an observation that hadn’t been reviewed by them, for better or for worse.
The main problem I have with this suggestion is simply how these points would be awarded. If user A adds adds a contradictory identification to an observation by user B, and user B changes their ID accordingly, then awarding points should be straightforward. If user B maintains their original stance, though, how would the algorithm know which user to favor?
A more difficult situation might be one involving four users: A, B, C, and D. If the first three have all agreed upon a certain identification, and user D contradicts them, then D would be overruled and the community ID would remain as determined by the first three. In this case, user D could easily be correct or incorrect. A similar situation might involve an observation made by user A with the other identifiers (B, C, and D) determining the community ID. Regardless of which side is correct, how would the algorithm know which users were more accurate in their determination(s)? Perhaps most difficult of all would be the scenario where user A (who posted the observation) is initially correct, but is convinced of an erroneous ID by user B. The observation would then reflect the identification suggested by user B, who would likely be seen as improving the accuracy of said observation and thus awarded points.
Don’t get me wrong, I really do like the idea. I just don’t see how it could be implemented successfully without an algorithm that already knows how to identify each organism, which would render the feature itself (and identifiers) unnecessary.
I feel that one of the easiest ways to avoid this situation (and make life difficult for overzealous identifiers) would be to award users with fewer points for simply supporting an established community ID. This would hopefully disincentivize just randomly agreeing to identifications on RG observations. As far as a badge system is concerned, I think that has some potential if implemented in moderation.
I’m sorry, but I feel like less can be more. And in this case, less complexity is more freedom. I’ve been around for a few major iNat changes and I never got along nicely with any of them :) Thus, I do not make many IDs anymore, I mostly upload observations because I am hoping they don’t change those anytime soon.
So long as you can accurately identify an organism and want to be the #1 ranked identifier (by whatever metric), who cares if you’re a global authority on orcas, or if you’ve just watched Free Willy a few times?
Haha, I wish there were more of these kinds of people. There are a handful, I could say their nicks in my sleep … but I feel like a warm welcome to iNaturalist with many identifications, even if they’re just stupid confirmations, is not a bad thing at all.
I feel that one of the easiest ways to avoid this situation (and make life difficult for overzealous identifiers) would be to award users with fewer points for simply supporting an established community ID. This would hopefully disincentivize just randomly agreeing to identifications on RG observations. As far as a badge system is concerned, I think that has some potential if implemented in moderation.
This sounds a bit like the system I proposed implementing about two years ago. But again, it means new complexity and most times will mean that your observation will be harder to get to Research Grade. I don’t care whether some crazy clicker makes my observation of a “Great Mullein” RG, or some other common species, I’ll just be happy it actually got to RG. Given there are so many observations on iNat these days, and so few IDers, we need incentivizes - not de-incentivizes.
I admittedly, sometimes use the iNat AI when I have a species that is out of my league. (NOT for identifying. Only for adding an initial ID to an observation I have added)
I think the iNat AI should not operate past a suggestion to the observer, and that wouldn’t add complexity. Just my opinion.
I see where you’re coming from, but the point of awarding users with fewer points for supporting an established ID is to prevent observations from having an excess of uniform identifications (generally more than three). A user agreeing to an observation which only has one identification should get the same amount of credit as before, but an identifier agreeing to something which is already research grade and has four or five identifications shouldn’t.
Ideally, this would still encourage people to confirm observations marked as ‘Needs ID’, but discourage behavior which may or may not be the result of trying to amass identifications. Perhaps there’s a way to encourage users to make confirmations (again, not in excess), although I’m not sure I know what that’d be.
To clarify, the suggestions I was referring to were those posted by the observer, not computer vision. From what I could tell, they were just clicking ‘agree’ indiscriminately.
Yeah, encountered that 500+, too, but found remedy very quickly. In the case of an OB of which identification I was not certain myself, I asked innocently, whether the “IDer” would comment on my doubts, him being the first on the leaderbord for the taxon in question. He has never returned to my observations again :-)
It is not always easy to do. For one, in some groups of organisms there is just no “bunch” of expert IDers or even just IDers. And quite a few serious IDers just feel distaste for this type of activity and prefer not to be bothered, feeling that this is a loosing game. I had such experience with an almost malicious user who was putting crazy IDs and overruling correct ones. I tried to enroll couple of experts and at least one of them wrote that he already had encounters of the said user and did not want to be bothered by him any more.
How would you see this working with withdrawn IDs? For example, I often put coarse IDs on unknowns. My policy with insects in particular is to take a best guess (provided I don’t ID to species level). If I ID something as lepidoptera, there’s a much better chance that someone will look at it than if I just leave it as pterygota. If - as happens sometimes - someone says it’s not a caterpillar, it’s a sawfly larva, or whatever, I withdraw my ID. My point is that the initial wrong ID is still providing a useful function, provided that I do in fact withdraw it later.
Ahhh… I understand that now. I still don’t see why many identifications is a bad thing, but I concur that agreeing to an observation with many IDs is not very useful.
Agreed, agreed! Even wrong IDs can help lead the way to correct ones.
Wow … interesting. I feel like kids and others who might mis-use iNat like this would be doing gaming or some introvert-type thing. Now, maybe I just don’t see them. I have agreeing-IDs off in my notifications :-)
Curators should give those people a warning. But making their agreeing identifications ‘less valuable’ on some scale wouldn’t stop any troll. You’d have to make an iSpot-like reputation system, which the staff have already expressed their distaste for (and frankly, I would not like to see that implemented either).
The mentioned IDer is not a kid ;-) Actually he presents himself as a serious expert. I started to doubt only after his 500+ IDs in various classes of a very large phyllum, where it is not possible to be expert or even moderately knowledgeable in all groups.
This is a good point. I´m also throwing stuff into specific families at times as I know it will be seen by someone who will tell me one way or another if it belongs there.
I think there´s many ways the algorithm could work to define “accuracy”, and whatever it was, it would need to take this and many other aspects into account. It certainly wouldn´t be able to be as simple a count as observations and identifications.
Maybe accuracy isn´t even the specific goalpost thats needed…just some measure of quality control…or a reward system for different actions which help motivate this.
There are some people who hardly ever give a first ID at species level - they mostly follow pre-existing IDs. I trust them less than those who dare to be leading with their IDs.
Ah yes, I liked it, so I must have seen it but forgotten.
I´d vote for this feature! Or some version of it.
My issue isn´t sooo much with the leaderboard chasers…I am in general, pro-gamification.
My issue is with the accuracy overall and the perception of the iNaturalist data by folks I interact with outside of iNaturalist. The noted 65% level accuracy in insects could be better!
I think gamification might help.