Remove ability for observers to agree with IDs on their own observations

[ralphma]

Hi everyone,

I realize this topic probably has been discussed extensively, but I’d like to raise it again for consideration. It’s common for observers to automatically confirm any identification added to their observations. If the id changes, they just adjust their confirmation as they don’t have any clue about the ID of the taxon - they just want to receive research grade. This behaviour isn’t typically a concern in well-documented regions like the USA or Central Europe with many active identifiers. However, in areas with fewer identifiers, such as Uzbekistan for example, this practice is very problematic. Here, observations often achieve research-grade status even if the sole ‘expert identifier’ isn’t entirely certain of its own identification. This situation likely leads to a significant accumulation of incorrect data, which, in my opinion, should and could be avoided.

To address this, I propose that observers should not be allowed to confirm an ID. If they are confident in what they have observed, they should upload it with the correct name initially. If they are clearly mistaken, they should have the option to reject their ID, but not to confirm an ID given by someone else.

What are your thoughts on this suggestion?

Best regards!

[traianbertau]

There are also observers who do research once a species has been suggested and try to verify/falsify.

BTW: There are also identifiers who blindly agree to ID’s not of their own observations.

[williampaulwhite17]

@traianbertau

There are also observers who do research once a species has been suggested and try to verify/falsify.

Yep. And god knows I’ve had enough old butterfly ID’s recently ID’d that I simply had forgotten about. I could have ID’d them now but not when I made them. But someone went through old butterfly records in Texas and ID’d them

[spiphany]

I agree that observers blindly agreeing with any ID that is suggested is a problem, but…

I’ve definitely had cases where I initially entered an observation with a more general ID that was refined by an expert and I later learned enough to ID it myself.

I have also occasionally entered a very stupid wrong ID (mistyping or misremembering the correct name) that was corrected by someone else, and of course once it was pointed out I corrected my own ID because I knew what the organism was, I just didn’t access my own knowledge correctly when entering my initial ID.

So there are also plenty of cases where it makes sense for an observer to agree with someone else’s ID.

One idea that has been proposed before which I like is removing the “agree” button from a user’s own observations. In other words, they would not be blocked from entering an ID that was suggested by someone else, but they would have to make the effort to type it in themselves. I’m sure some people would still blindly agree, but it seems like this would encourage users to think at least a little about what they are entering and reduce the likelihood that people treat it like a “thanks” or acknowledgement, as many users seem to do with the “agree” button.

[lj_l]

@ralphma

If they are confident in what they have observed, they should upload it with the correct name initially

Are you suggesting you shouldn’t be allowed to change your initial ID’s? What if someone learns how to identify a taxon later? Or if someone feels like they made an incorrect ID, without anyone else identifying the observation (if they are only allowed to withdraw, wouldn’t that leave the observation at “life”?). On multiple occasions I’ve had to re-ID my observations, often to the ID of someone else, because I’ve gained more information than I’d originally had.

Additionally, I might post an observations at “spiders” or “fungi” or even “life” if I’m not sure, and then research later. If someone then confirms the observation, and I come to the same conclusion, would I not be allowed to agree?

I agree observers blindly agreeing to ID’s is a problem, but I don’t think not allowing the observer to add a new ID to an observation is an ideal solution.

[tiwane]

I don’t think this is something we would ever do. It removes agency from one’s own observations, and doesn’t give you a chance to learn and do your own research and agree. I think it would also increase incentives to create sockpuppets.

What I’m for would be pop-ups for new users the first few times they hit “Agree” that provides an explanation for when should agree or shoudlnl’t agree. I think the suggestion that @spiphany also brought up would be a potential option.

[whitneybrook]

A lot of times I only ID to genus when I have doubts or want to avoid the auto agree problem. However, I feel like I should be able to add an ID I think is right, even if I’m not 100% sure. Having the CID at species could increase the chase of someone comfortable IDing it to species finding it.

Just this morning I started a new strategy. I added a comment: “I think we could use another confirming ID. Moving to Needs ID for now.” Then I selected “Yes” for “Can the Community Taxon still be confirmed or improved?” The comment is partly to remind me to update my DQA when if/when get the confirming ID.

I’ll see how it goes. I likely won’t do this for species where I have a high degree of confidence due to the increased work it creates.

[jimmoore]

Bad idea, as others have said. Among other things, when an expert identifies something, they will sometimes be the only one on iNat with the knowledge to confirm the ID. So the only way it will attain research grade is if someone else confirms it–and it should count as research grade imo because an expert has identified it, and their opinion should count as least as much as two non-experts identifying something.

In any event, iNat’s investigations to date suggest that the current research grade data is c. 95% correct, so I don’t think there’s a big problem that needs to be addressed.

[spiphany]

@jimmoore

Among other things, when an expert identifies something, they will sometimes be the only one on iNat with the knowledge to confirm the ID. So the only way it will attain research grade is if someone else confirms it–and it should count as research grade imo because an expert has identified it, and their opinion should count as least as much as two non-experts identifying something.

I strongly disagree with this. Just because someone is an expert doesn’t make them immune to mistakes. I’ve seen plenty of observations where an expert made a mistake, someone agreed to their ID merely because they were an expert, and then multiple other IDers were required to get the observation back on track.

If there is only one person on iNat with the knowledge to confirm an ID, the solution should not be that their ID should count more than other people’s, or that it is OK to agree so that the observation becomes RG. The solution should be to figure out how to get more people on iNat who have the relevant expertise, whether it means recruiting specialists to join iNat or that other users working to acquire the relevant expertise, possibly with the help of the existing expert.

@jimmoore

iNat’s investigations to date suggest that the current research grade data is c. 95% correct, so I don’t think there’s a big problem that needs to be addressed.

This really varies depending on the taxon; the assessment looked at all observations, including ones that are easy to observe and ID. There are some taxa where the majority of observations are incorrectly ID’d.

As someone who is working on learning a difficult taxon where there are not enough IDers, the knowledge that many users will blindly agree to any IDs I suggest, no matter how tentative, makes me more hesitant to suggest species level IDs – because it is quite likely that if I am wrong it may be months or years before anyone else looks at the observation and corrects it, and if the observer has agreed, that wrong ID will have become RG and shared with other databases.

[frontyardscientist]

I think educating observers about the risks of automatically agreeing with IDs made on their own observations, as I and many other identifiers have been doing, is the best course of action if iNaturalist is to remain a democratic platform. In my opinion, it makes sense for observers to have control over their own observations.

• Any identifier irked by an observer’s systematic agreement should simply withdraw their ID to revert an observation to Needs ID and restore it once another identifier comes along to identify the observation.
• Preventing observers from pressing the Agree button on their own observations would not prevent them from manually submitting agreeing IDs.
• Many observers, myself included, agree with IDs made by others on their own observations only after thorough verifications/research on their end. If those should be allowed to press the Agree button, so should all observers!

[jeanphilippeb]

As an observer, if I put a high rank ID on my observation (because I don’t know what it is) and if someone else put an ID at rank species, likely correct, either I let my ID unchanged if it does not disagree, or I just remove it if it disagrees with the likely species ID.

As an observer, if I put an ID at rank species on my observation and if someone else put another ID at rank species, likely correct (because more knowledgeable, or because there is a convincing comment), I just disable my ID.

So, with my current practice, I see nothing wrong with the feature requested. If you are that much concerned about observations reaching the status RG, I would just say I have many observations that you could review, for instance this one (whose ID is correct, with confidence as high as it can be, still waiting for a 2nd ID after 5 years).

So, despite all other explanations above, I vote for it.
@ralphma, don’t you vote for it?

[s-e]

My 1st reaction as a new iNat user was that 3 IDs to reach research grade would be much safer (both because of the possibility of this automatic agreement, and the risk of AI/VR error for the initial ID), but as we can see, when it comes to some taxa, identifiers are seriously lacking, even by tagging those listed for the taxon, you never get 2 IDs and RG… which could be a loss for “science” when an observation is unique for a country for example, or out of its known range.

[jimmoore]

Experts aren’t immune to mistakes–of course; I never suggested otherwise. But so what? They are much more likely to be right than non-experts, so logic dictates their opinion should count more.

As for getting more experts in a field on iNat, good luck with that. And in some cases, an expert may be the only person in the world who has studied a particular group. Moreover, just adding more knowledgeable people doesn’t necessarily improve anything–multiple knowledgeable people can make the same mistake a single person does.

[kevintoo]

It is easy to overestimate the risks of people ‘automatically’ agreeing with the first ID. New users observing easy-to-photograph, and identify, species are most likely to do this. An expert is not needed to fix those situations.

[spiphany]

@frontyardscientist

Any identifier irked by an observer’s systematic agreement should simply withdraw their ID to revert an observation to Needs ID and restore it once another identifier comes along to identify the observation.

This unfairly puts the burden for the observer’s ID on the identifier (the IDer has to follow notifications for agreeing IDs, withdraw their ID, restore their ID, etc.), even though this is not really their responsibility. The person responsible for the agreeing ID is the person entering that ID (i.e., the observer). Many observers do not appear to take this responsibility seriously or realize what they are doing when they agree to an ID.

It’s great that you verify/research IDs before agreeing; my experience is that most observers who agree do not in fact do much if any research. (This is based on the fact that the agrees come quickly, often with in minutes or mere seconds of receiving the notification; they never ask questions, even if their original ID suggested little knowledge about the taxon in question; they agree to IDs that are non-obvious, such as a taxonomic level like tribe or subgenus that is unlikely to be mentioned specifically in the literature; they agree to IDs made by anyone, regardless of how plausible that ID is or whether the person who provided the ID has any experience with the taxon, etc.).

@kevintoo

It is easy to overestimate the risks of people ‘automatically’ agreeing with their own IDs. New users observing easy-to-photograph, and identify, species are most likely to do this.

This is not my experience, unfortunately. I see lots of experienced users blindly agreeing to IDs on observations of insects that are frequently observed but difficult to ID.

@frontyardscientist

I think educating observers about the risks of automatically agreeing with IDs made on their own observations, as I and many other identifiers have been doing, is the best course of action

I would welcome suggestions about effective ways to do this. My experience is that whenever I have tried to suggest to observers that they should not agree to an ID they cannot verify themselves, this has had little or no effect on their behavior, either for the observation in question or future observations.

So I do think something else is necessary to encourage different behavior. Asking observers to type in the ID they want seems like it would be a start because it would at least require observers to think about what they are doing when they enter an ID. I fail to see how this would be unfair to those observers who do research their IDs – if you know what it is, you should be capable of typing in the ID.

@jimmoore

Experts aren’t immune to mistakes–of course; I never suggested otherwise. But so what? They are much more likely to be right than non-experts, so logic dictates their opinion should count more.

And how do you propose to measure whether a given individual is in fact an expert on the particular taxon in the particular region? How will you determine whether any given ID is within their area of expertise or outside of it? How do you know that that expert was not tired or distracted or in a hurry when entering the ID? For difficult taxa that require special skill and knowledge, the possibility that even someone who is an expert might make a mistake is greater than for easy taxa. Just because they may be more likely to be right than a non-expert does not mean the possibility of them being wrong is zero.

Someone else’s expertise does not justify adding an ID that you cannot verify yourself.

@jimmoore

Moreover, just adding more knowledgeable people doesn’t necessarily improve anything–multiple knowledgeable people can make the same mistake a single person does.

So? To use your own argument: logic dictates that multiple knowledgeable people are less likely to all make the exact same mistake than a single knowledgeable person plus a non-knowledgeable person who is blindly agreeing to the ID.

[kevintoo]

@spiphany

This is not my experience, unfortunately. I see lots of experienced users blindly agreeing to IDs on observations of insects that are frequently observed but difficult to ID.

You miss my point. That situation is far less common than new users just hitting agree.

[spiphany]

I did not miss your point. A large portion of the users I see hitting agree are not new users. I don’t consider the situation uncommon at all.

I also don’t think that blind agrees for observations of difficult-to-ID species are a negligable problem even if they are outnumbered by observations of easy-to-ID species. The consequences of a wrong RG ID for an uncommon or difficult species are more significant than a few mis-IDs of common species.

[frontyardscientist]

@spiphany

This unfairly puts the burden for the observer’s ID on the identifier (the IDer has to follow notifications for agreeing IDs, withdraw their ID, restore their ID, etc.), even though this is not really their responsibility. The person responsible for the agreeing ID is the person entering that ID (i.e., the observer). Many observers do not appear to take this responsibility seriously or realize what they are doing when they agree to an ID.

Perhaps I’m just a pushover, but I’ve personally assumed the responsibility of following up on my IDs, so it doesn’t seem unfair to me. However, I hadn’t considered that the method I proposed required enabling notifications for agreeing IDs, which I have not done because it clogs my notification feed. With that in mind I agree that it would be a huge chore to withdraw all IDs with which observers have systematically agreed.

I think data on the frequency of systematic agreements, the kinds of observers responsible for them (number of observations, identifications, last date of activity, etc.) and the taxa most affected would help assess the objective gravity of the problem and develop solutions if deemed necessary. Without data all we can discuss is our subjective experiences, which while valuable do not necessarily correspond to the experiences of most identifiers.

An explanatory pop-up as suggested by tiwane seems like a good solution. It would probably take 2-6 months to implement whereas educating observers is relatively easy (albeit time-consuming) and immediate. I agree that a long-term solution should eventually see the light of day though

@spiphany

So I do think something else is necessary to encourage different behavior. Asking observers to type in the ID they want seems like it would be a start because it would at least require observers to think about what they are doing when they enter an ID. I fail to see how this would be unfair to those observers who do research their IDs – if you know what it is, you should be capable of typing in the ID.

I agree. On a partially related note I think the ultimate solution is MORE IDENTIFIERS to check all the RG obs that result from systematic agreements. Of course, iNat will never have quite enough identifiers to keep up, because let’s face it, accurate IDs are hard for most taxa and iNat itself is a niche platform/interest. It’s already so difficult to recruit observers outside of bioblitzes, and only a fraction of bioblitz participants stick around, of which a smaller fraction take to identifying others’ observations

[igor117]

I support this, but unfortunately there is no chance that it will be approved. Most users and devs don’t really care about quality of data, at least it’s clearly not a priority.

[bbk-htx]

@ralphma

It’s common for observers to automatically confirm any identification added to their observations.

How can one determine if it was “automatically” confirmed. I can’t quantify what is in someone else’s mind.

@ralphma

they just adjust their confirmation as they don’t have any clue about the ID of the taxon - they just want to receive research grade.

There is no way to tell looking at the observation if this is true or not. I do not know what is in someone else’s mind.

If I am certain, and I use the “agree” button, I can quickly move on to the next…

I think that any serious researcher would draw there on conclusion about the observations they are using whether it is “Research Grade” or not.

“Research Grade” is not a be all, end all.

Sorry to disagree.
More education will lessen the need for more legislation.