Implement an iSpot style reputation system

I don’t get the impression that you want to take part in a discussion. You have an opinion that no amount of dialogue will likely change.

As a user of this site, I want my observations to be identified with as much expertise as possible, whether that be from knowledgeable “amateurs” or from true taxonomic authorities. When I upload an angiosperm, I do so knowing that any ID I give it should count for far less than someone who truly knows the group’s biodiversity and the diagnostic criteria needed for accurate identification. I’m not ashamed of being an amateur when it comes to flowering plants and I certainly wouldn’t presume that my ID should count for much of anything. That being said, were I to put in the effort to learn the group… to provide accurate IDs for others… I would expect my “reputation score” to increase proportionally. Seeing those statistics might encourage me to learn more and eventually become something closer to a true expert.

My concern is that all the benefits of a reputation system on iNaturalist will be ignored because of comments like yours. Nobody can be an expert in everything. There is no shame in having less knowledge than others.

4 Likes

Opting out of community ID doesn’t bring the observation to research grade unless it would have been research grade without opting out.

3 Likes

“Knowledge” is a very difficult thing to measure. Things like numbers of papers published or level of higher education are largely influenced by various cultural, economic, and social biases and may be affected more by that than usefulness as an inat identifier. Conversely, volume of IDs or popularity amongst other inat users are also subject to bias. I think the current system works well for the most visible and often observed taxa. More technical taxa are hard anywhere under any circumstance. And this is meant to be an open community for a broad audience not another replication of the existing academic system with all its upsides and downsides we all well know.

4 Likes

The bigger issue seems to be misidentifications that linger on in the range maps. I can un-research grade an observation, but I can’t fix the range map errors introduced by an opted-out misidentification. There have been at least a couple forum posts on this recently.

https://forum.inaturalist.org/t/what-to-do-about-opted-out-of-community-id-when-identification-is-wrong-and-user-is-inactive/6790

With enough correcting ids the points will stop displaying on the range map. I think the initial wrong id needs to attain “Maverick” status

2 Likes

And this is meant to be an open community for a broad audience

Then stop using the term “research grade”.

As I see it there already is a reputation system, but an invisible one: there are certain taxa that some individuals have dedicated a large amount of time to maintaining in an error-free state. This is not something that anyone outside of the site would have any way of knowing. “Research grade” for spurges means something totally different than a research grade dipteran.

3 Likes

I completely agree with the first part. There are many ways to be an expert. A regular user who consistently identifies correctly should have a voice similar to a vetted taxonomic expert. (Though I would argue that a true taxonomic authority should have curatorial powers that trump community IDs. For example, Anthony Gill, the world’s expert on dottybacks is a regular here… nobody should be able to outvote his determinations.)

But I strongly disagree that the current system “works well”. I’m sure it’s adequate for giraffes and flamingoes, but not for Sarcophaga and Tridacna. If it worked well, there wouldn’t be such a loud chorus calling for this change. Of course, I don’t doubt that there will be an equally loud chorus crying foul.

Go back and read the whole sentence and you’ll see we are basically saying the same thing.

I’m not sure where I stand on this issue. But honestly, iNat could potentially benefit from this system. A lot of my reasoning here also matches the discussion on why I will not use opt-in for my observations.

So I could see the reputation system “fixing” the opt-in system in part. It would solve one main issue for me, which is users that either use AI ID because “it is the top option”, guess, or don’t have enough research to make the ID they make. This is mostly an issue with newer users, from my experience, especially school groups (which is another subject in itself). I think this could be solved by a reputation system – preventing newer users or those without basic ID experience from agreeing with their school friends or just observations in general and just hitting RG immediately.

I’m theorizing that this could arguably lead to a higher data quality on iNat. But besides this? I don’t see what a reputation system could accomplish. It might label experts which users could then call to ID observations, but we already have “Top IDs” serving that role.

So my interpretation of this in a working iNat sense is not “marking experts”, but rather “clearing new users” or users who do not usually ID certain groups. It could be really basic. Instead of various levels of expertise which is what most of us are debating, earned by accumulating “correct IDs” over time, it could just be a simple entry level requirement. Make 25 correct IDs on plants in the US, and you are “cleared” for that region, and IDs work as normal. Less than that, and your IDs have less weight than normal (e.g. requiring more than 2 for RG).

8 Likes

Taxonomy may not be a democracy, but it is also not an objective truth merely a set of hypotheses that have been refined over hundreds of years. There isn’t always a right and wrong answer to what a species is or isn’t, and even experts may not agree with each other.

8 Likes

This shouldn’t be the case. Even the number one expert in the world makes mistakes. What if Anthony made a mistake, but the hypothetical number 2 expert in the world (or no. 3, 4, etc) wanted to correct him?

5 Likes

Also, possible structures for experts in a taxon are clear, but how would you treat ‘experts’ per se in an area? As an Australian example, the user @ellurasanctuary (run by Brett and Marie Smith) isn’t an expert in a taxon, but in fact an ‘expert’ for South Australian (the state, not the region) species across the board. How would that be weaved into a reputation system?

2 Likes

I was perhaps being a bit hyperbolic there. No ID should ever be set in stone. The sentiment is that it should take tremendous effort to overturn the ID of an actual taxonomist on here. But obviously taxonomic concepts change over time and observations will regularly need to be amended. I would expect that any disputes would be easily resolved if there are multiple experts around to weigh in.

I fully agree here, but I think one of the biggest hurdles is that for a lot of taxa, there’s only one expert on iNat.

1 Like

That’s not what I wrote. I very clearly in the kickoff message noted the ability of users to get additional votes or points or whatever you want to call them through contributions . Although I would love to hear how on a site with hundreds of thousands of users, anyone could ever do enough leading ID entries to promote themselves unless the thresholds were very low.

And, additionally as an ispot user confirmed later in the thread that system did go from 1 to 1000, so how exactly am I proposing a ridiculous system?

Sorry if this is the wrong place to ask this, but I clicked to vote button and it say I voted, but I don’t know what I voted for, I thought it would give me options, I am afraid I voted yes, but to me this don’t sound like a good idea.

1 Like

To remove your vote if you so wish, go to the top of the thread, it should say voted at the top left under the number of votes, click it, and it should display an option to remove your vote.

3 Likes

This sounds like a bad idea. First off how will we have enough approved experts to identify everything in a reasonable time? And how do we decide what taxa to consider someone an expert in? What about all the good id’s here that were made by amateurs? If we have a problem with bad id’s we should increase the number of id’s to make it research grade from 2 to 3 or more, not make it so only approved uses can ID

The inefficient system where only a few approved users (curators) can reclassify things is one reason I stopped using bugguide, this site is much better, ant seems pretty accurate even as it is, and even for relatively obscure things like insects

That said, there are some taxa where I think AI shouldn’t give species level suggestions, as this may be leading to issues like things being listed as Polistes fuscatus (which AI suggests because it is the most common dark polistes) when the picture isn’t close enough to have a chance at distinguishing from P.metricus or P.parametricus

7 Likes

I suspect that the success of iNaturalist is due in part to the egalitarian nature of membership. I certainly was encouraged to assist with identifications knowing that my opinion was as valued as anyone else’s opinion. I was also keenly aware of the need to identify with care because I carried an equal weight to others. All collections have their identification errors, and there are "…tools for assessing the data quality of each record,…"¹ Perhaps part of the secret sauce to the success of iNaturalist as a citizen science site is the egalitarian nature of membership as a citizen, both as an observer and an identifier.

I believe any form of reputation system would change the core ethos of the site. I also suspect that given the global reach of the site and the sheer number of users, any implementation would be sufficiently complex as to have multiple issues of its own, and many unforeseen consequences.

15 Likes

I feel like this would create quite a bottleneck at first - until enough people have earned the reputation to help bring things to RG, how would the small amount of designated experts keep up with the 20000+ observations coming in each day? This would create a huge backlog of needs ID observations with correct ID’s, many of which will have been confirmed by people who know how to ID that taxon, but haven’t had the time to earn enough reputation.

3 Likes