Issue with users automatically agreeing to an identification

I am not sure where it is best to post this topic:
I noticed that a lot of users initially post an observation without identifying it in the first place (which is fine), but as soon as an other user suggest an identification they (dumbly?) agrees to it.
So we can end up with research grade observations with very few validation and a wrong ID.

I do not know how it is best to deal with that behaviour; perhaps iNat should raise a confirmation message when the same user propose (or agrees to) an ID that is different of what he proposed earlier. Also I would recommend that at least 3 different users suggest the same species ID before it go to research grade status.

Thank you.

6 Likes

I see this often, what I do is advise the person nicely that it is not always wise to immediately agree with an ID, I have had great success doing that. Or you can always just tag someone who will know for a confirmation. But yes, it’s probably a good idea if iNat can possibly set up a system that helps decrease this, I can’t think of a way that won’t annoy proper identifiers though.

3 Likes

This is quite the issue. I do a lot of Salticidae IDs, which require counting hairs, looking at eye ratios, or any other very subtle sort of distinguishing. I’ll bring the ID from Family to Species and immediately have the user agree with my ID.

Maybe if we as identifiers had some disclaimer to copy/paste such as the words written in the ‘help’ section under identifications on this site: “Please do not simply “Agree” with an ID that someone else has made without confirming that you understand how to identify that taxon. An identification confirms that you can confidently identify it yourself compared to any possible lookalikes.”

Having a copy/paste at the ready would be very useful, especially if you ID like I do… dozens or hundreds of IDs in a day.

4 Likes

Maybe, instead of using the word “Agree” use the word “Confirm ID”… might make people think twice…

5 Likes

I wonder if this is any different than an observer blindly picking the first species that is automatically suggested and guessing “right”, or an identifier blindly agreeing to an observer’s id. I’ve noticed the latter happening as well and that results in even more inaccurate ids than the situation described above. If a changed id by an observer is discounted, then perhaps observers shouldn’t be counted among the two ids. Or rather, there should be at least 3 ids. That would cover all bases. An immediate change of id may simply be because the observer was debating between two possibilities and views an identification by a more knowledgeable person as confirmation of their alternative possibility. It is no different than if they had proposed that alternative to begin with.

3 Likes

It would be a problem if ResearchGrade was irreversible, but misidentified RG observations can be corrected, so I don’t really think it is too much of a problem in reality. RG obs aren’t harvested immediately anyway by data partners (possibly in part for this reason). The onus is on identifiers to be vigilant for misidentified RG obs.

9 Likes

I don’t see this as a problem. RG observations are meant to be used by researchers. When going over these we can expect them to check them individually to make sure they are correct and to discard the iffy ones. These users may also feel comfortable trusting the IDs because the identifier has a lot of identifications or the site calls them a top identifier.

5 Likes

I do not think that you can expect researchers to go over each one individually to check… but you might be able to expect them to add some uncertainty into their calculations to account for misidentifications (I do at least).

2 Likes

You’re probably right. I had to go over observations for a paper but there were less than a 100. It would probably take too long for someone working on a different taxon

It makes no difference to big number crunching research anyway. The more obs are used, the less effect of a few misidentifications.

This is an issue, along with blindly accepting the first suggested ID, where I am. If it’s in an area where there are a lot of regular users the errors get weeded out, but it you’re in an area where the users are transient and iNat doesn’t have enough observations over-all to provide good recommended IDs (and also where the IDs are complicated and people make incorrect IDs often) it’s a real challenge.

I’ve introduced my staff to iNat and have had to spend a of time explaining this issue to them as iNat often suggests IDs for species from a completely different part of the word and users will do the same when making identifications. I’ve told my staff to double check the suggested species range and to cross-check with the species lists we have for the area.

That helps, but it’s not a solution that works in the majority of situations.

For the greater number of users I’m no sure what a good solution is, but I do think it’s an issue that needs more attention.

3 Likes

Personally, I think the ‘agree’ button should be removed. That would cut down on this sort of behavior a lot. From a psychological perspective it’s very easy to off-hand hit agree, but less people will go to the trouble of putting in the species name in to their own suggestion unless they are sure.

The agree-button as it is seems like it’s asking a question, and psychologically people are inclined to answer.

12 Likes

It has been pointed out how “Agree” button IDs in the thumbnails of the Identify page don’t show descriptions or dialog, so often aren’t seeing crucial info relevant to an ID.

New users typically see “Agree” as a form of Like or a thankyou to an identifier, so much more on-boarding about that needs to be done. We try and educate new users when we encounter it, but…

I will often confirm an ID, not fully knowing for sure… but only if I have come to trust an Identifier’s accuracy and knowledge. I consider it a form of adding weight to an expert, but I don’t give such weighting “just because they are an expert”, I have to see and experience and develop that trust. I won’t add the weight where there is any sort of dissention however, and if dissention develops, I will often pull or withdraw my supporting ID.

IDs are dynamic, as Stephen pointed out, so CID will eventually settle in the right place. There are many of us that go through and review ALL obs (well… to a filter for place and/or taxa) including the RG and casual ones, and we pick up many of the mis-IDs. If you are a specialist or expert, or have resources that could make you one, then doing likewise for an area of observations can add value in a very real way.

And it is Community ID, so you don’t need to worry too much about them being “wrong IDs”. Make your ID as you see it, start dialog if you think it will help others learn or teach, and then let CID fall where it may. You can tag in others that you think might contribute their views, but avoid the temptation to “fix” wrong IDs by gaming the system. We’re not voting in a new Government here…

I’m certain that iNat staff and developers are aware of these issues, and will do what is best for the future direction of iNat.

2 Likes

I’m not sure what type of research you are referring to but I don’t think most field biology research gets into the “big number” category. And much of it deals with ranges, so misidentifications that erroneously represent where it occurs are problematic.

And, I agree, that automatically agreeing is a problem. Even if an expert IDs it, before agreeing with that ID, it would be best to investigate to see if that seems right. That way you learn a little more about the organism and experts aren’t infallible.

2 Likes

Guilty!

For sure, I understand that this makes the data less scientifically valid, but I try to follow some kind of process for Agreeing other users’ identifications of my observations.

Firstly, what do I know of the identifier’s expertise? I’m not going to doubt a beetle identification by borisb, for example.

Then, what was my original impression? I also realise this is not always watertight, but I try to do some research into species I am unfamiliar with to get some idea, but then leave my own identification for the observation at “Class”, “Order” or “Family” level if I have no real knowledge to back this up!

Next, I (almost) always take a look on-line to see how the new identification compares with information available from web resources. Often, I have no idea how close one species may be to another nor how to differentiate them, but I’ll Agree anyway if it looks right!

I think for new users it’s very difficult to know if other users’ identifications are accurate, and the desire to learn new species and find out what one has seen can override scientific integrity, but this also leads to new learning, and corrections are always there to be made.

On the other hand, I also agree that any scientific use of the data should require extra validation by the scientist, depending on the context. Observations of Eurasian Magpie, for example, can surely be taken at face value without too much scrutiny, whereas observations of Grey Dagger moths should be taken more cautiously.

3 Likes

There’s another embedded issue here. As it stands now people can delete their account and remove all their IDs. If they do this and you haven’t agreed to their ID, you lose it completely. So unless they somehow make the community ID retain the deleted ID you have to do this if you want to ensure the ID remains. So I’m torn… but I do this sometimes when I otherwise wouldn’t because of that factor.

4 Likes

Me too. Also, I think new users especially sometimes think that getting to Research Grade is so important that they want to reach that goal as soon as possible, or help their friends reach it. Maybe de-emphasizing Research Grade would help.

1 Like

I like the suggestion by @indignantchickadee: 3 IDs for an RG observation.

5 Likes

I came here with a similar reply in mind. It can be hard in some cases to tell the difference between thoughtless reactive “agree” button hitting, vs. a quick review of an unfamiliar ID provided by another user and subsequent willingness to confirm.

This morning I added an observation of a water bug- I am no Hemiptera expert so I left it at the infraorder for “Water Bugs.” Another user quickly added an ID to genus, so I clicked on that genus, browsed through the info and species, checked a couple related genuses, and feel comfortable confirming that the observation belongs in the suggested genus and is probably one of just a couple species in that genus). So I hit the agree button.

https://www.inaturalist.org/observations/24781068#activity_comment_2874194

All of this really just took ten minutes or so, so if I hadn’t left additional commentary on candidate species along with my “agree,” it would be easy to think that I had just automatically hit the “agree” button. So while I agree that there is a widespread problematic behavior here, there’s probably also a whole spectrum of increasingly diligent behavior that’s not necessarily possible to tell apart from the problem behavior at a glance.

4 Likes

What is the recommended way to disagree with an ID suggested by an observer? I ran into this quite a bit recently.
Is there any way to encourage comments to show why one agrees with an ID.
I had the window come up a few times that questioned if I was sure about the ID or not. It made me pause to go over why I either Agreed or Disagreed.

1 Like