Agreeing with experts and "research grade"

As a sort of mid-level iNat user (neither novice nor expert identifier), I do one of the following when someone adds an ID to my observation -

  1. If it’s something I know quite a bit about, or an ID that I had a hunch of (but didn’t add because of my uncertainty and probability of biasing others’ inputs) then I will agree with it after a very quick, cursory Google search.

  2. If it’s something I know a little about and can do with a little more expertise myself, I will ask the identifier to point out the key characters, then agree to it once I know what I see. I may tag another identifier if I’m still confused.

  3. If it’s something I’m absolutely clueless about (to the point that additional info about the ID will be of little use to me), I’ll thank the identifier and tag another one from the list of top identifiers for that taxon, available on the right hand side. I will keep an observation of this kind from turning RG from my agreement, even if the first identifier is very confident.

When I’m the identifier of an observation and my input is based on a very strong but non-100% guess, I add the ID with a sentence asking the observer not to agree unless they are basing it on other strong inputs (either via other identifier(s) or via their own background reading).

Also, this is very directly connected to what RG means, or should mean - https://forum.inaturalist.org/t/rename-research-grade-discussion-and-polls/590 (you might have seen this already). I sometimes find myself wishing I could express agreement with an ID, without having the action push the observation to RG. :) I guess that’s a stretch, though.

3 Likes

That’s correct, but what is and isn’t “Research Grade,” and the choice of terminology, is somewhat arbitrary, e.g. why is it a greater than 2/3 majority and not a greater than 5/7ths majority, and why do we call it “Research Grade” and not “complete and probably accurate and we’ll share it with some other websites if they want it I guess”?

If an expert identifies my observation and I know they’re an expert, then from my perspective, it is identified. It would be nice if it got to GBIF, but I don’t care too much if it does. I’m happy to learn a name, and maybe begin to see the pattern that name describes. If my own identification is somehow contradicting the expert’s ID, e.g Mark says it’s Castilleja and my previous ID was Triphysaria, I’ll usually withdraw my own ID.

9 Likes

I’m with this guy. (And props for acknowledging the wizard that is @markegger!) I have seen posts with a chain of confirmations, and as soon as someone makes a different ID the whole chain follows suit. We discussed this elsewhere recently, but I still am very suspicious of that and attribute it to what sociologists call “confirmation bias”, basically that humans tend to favor agreement with a claim over disagreement.

I get the motivation to make things Research Grade. I want 100% of my observations to get there, and nearly as much for my region. Sometimes I don’t post things I’m not sure will get to Research Grade, not on purpose, but because of that mysterious inner compulsion. However, I don’t think it is to anyone’s advantage to compromise accuracy for agreement. Better accuracy makes the data more valuable. It’s not useful for reference or research to pull all of the RG posts of a species and find a bunch of them are incorrect, and erodes the credibility of the whole project.

I’m in favor of a conservative approach, to ID as far as I am able with the reference material at hand. Sometimes an “expert” is a part of that reference material, but with caution, because there is a level of scrutiny and effort (called scholarship) that goes into the creation of a species description, that is not universally applied by an expert scrolling through their iNat feed, looking at cell phone photos.

The counterpoint is that taxonomic work is imperfect and incomplete, and reference material only gets you a certain kind of knowledge. I’m not arguing that you shouldn’t ID unless you are somehow free of all doubt, and certain of the validity of the taxonomic structure and description. You only know what you know, and we are all still learning, so we ID away as best we can and should be satisfied with that. However, before agreeing with the proposed ID, one should at least do some due diligence to learn about that species and rule out similar-looking ones before clicking “that big green button.”

2 Likes

I’ve been part of posts like this, and I don’t see the problem. Person 1 identifies something as Sugar Maple, three other people take a quick look and agree. Person 5 comes along and identifies it as Norway Maple - “look at the attenuate leaf tips, that’s a Norway”. Persons 1-4 return to correct their identifications as well, either having learned a new character, or getting a reminder to take a closer look at the next maple they see. That’s a good system I think.

6 Likes

Hmm.

I still wouldn’t agree with someone just because they’re an expert. In the case you gave here, when the person ID’ed it, I would use it as an opportunity for me to learn how to do the ID properly, and then if I felt able to do so, I would agree.

Otherwise, I would say: “Not today.” And I don’t think this is crazy.

When I click “agree” I feel that it is making a claim that I know how to identify the organism, a claim about my own knowledge. If I truly do not know how to do it and the only reason is that I trust another person’s authority, then really, me clicking “agree” is misleading. The field was not designed as a testimony to the other person’s authority.

If someone truly is an expert like this, why do we even need multiple ID’s? And why would their ID be able to be “overriden” if a bunch of other users came in and “agreed” on some other ID?

Yes, it does. But I think that the bigger problem on iNaturalist is by far things incorrectly ID’ed, marked as “research grade”. As an example that came up the other day, I found quite a few plants mis-identified as Pokeweed, far outside Pokeweed’s native range. But I also found a few plants that I identified as Pokeweed, outside its native range. iNaturalist provides a way of tracking things like the expansion of a species…but what good is it if it’s bad data?

I think the data would be far more useful if people erred on the side of caution in ID’s.

4 Likes
  1. In the short term, it may be as good as an ID you’ll get, but over time other experts will find your observations and others will become educated on how to ID the group in question. Physical specimens in herbariums and insect collections can sit for years and years before being sorted and identified by relevant experts. It would be great to have way more experts on iNaturalist, but quickly agreeing with the few experts that are on iNaturalist does not solve that problem.

  2. If an expert is really the only person who can ID something and that expert isn’t publishing or sharing objective diagnostic criteria that allows others to make identifications then their expertise means nothing. Science is about sharing information.

  1. Exactly. If an expert IDs your observation, that’s great, you have your observation ID’d to your satisfaction. At this point, ask yourself what you gain by clicking “agree”. You’ve gotten your answer from the expert and it’s been recorded. I trust the experts too! I just don’t click agree unless I review some keys or something to at least rule out similar species.

  2. We need better guides and keys and these need to be shared more broadly. It would be cool if this could be built into or linked from iNaturalist taxon pages: https://forum.inaturalist.org/t/add-an-interactive-system-to-glean-diagnostic-features-from-identifiers-and-show-them-to-observers/3225

5 Likes

I’m not an expert by any means, but I will say this much. Sometimes, even experts in the field make mistakes. I’ve corrected a number of them on iNat, BugGuide, and other entomological websites that have species of planthoppers.

4 Likes

Yep, trust, but verify.

4 Likes

In addition to the fact that experts can make mistakes, many (most?) of the people doing lots of identifying on iNaturalist are interested amateurs, not experts. Lots of them are in the process of learning how to identify the subjects better, which means they’re making some mistakes. This is super helpful for learning, but only if the mistakes get caught, which is less likely if the observer just agrees with the ID and makes the observation RG.

11 Likes

This seems to be converging with another recent thread (https://forum.inaturalist.org/t/issue-with-users-automatically-agreeing-to-an-identification/2987/48).

I’m fairly new to iNat, but I’ve been trying to ID a fair bit in my area of expertise. I have to say it does irritate me when an observer, who clearly had no idea about an ID, quickly agrees with me at species or subspecies level. A number of people in this thread have explained how they do their own checking before agreeing with someone else’s ID of their observation. I do the same, but I really think we’re in a minority. Simply looking at the speed with which many observers agree with my ID tells me they can’t possibly have had time to do any checking themselves. As we all know, the end result is that we end up with a lot of RG obs that have only one ‘reliable’ ID (and if that’s wrong…). That potentially degrades the dataset, and in effect defeats the purpose of the requirement for two IDs to reach RG.

In the other thread, requiring 3 IDs for RG was suggested as a remedy, but didn’t seem to be favoured by many (it wouldn’t be a problem for bird IDs for example, but for groups with few experts it probably would). Removing the Agree button for new users was also suggested, but I don’t think that would solve the problem—it’s very quick to type in enough of the suggested ID to make it come up.

I suggested stopping an observer who had entered an initial ID from changing that ID later (for that obs only), effectively preventing them from agreeing blindly with the next ID provided (see the other thread for a few more details). That didn’t go down well with the frequent users who, it seems, routinely upload obs with no ID or high-level IDs, and come back later to ID them.

This seems to be another argument for the ability to load obs in ‘draft mode’, then ID/edit/add to them if you want to before they go live. If that could happen, stopping people from changing their initial ID could be implemented, and the frequent users could still change their own IDs in draft mode. I suspect many of us would like the dataset to be more robust, and perhaps this would help. And who knows, realising that they can’t change an ID later may even make some observers spend a little more time on deciding their initial ID.

Does this mean that if I photographed a random beetle, submitted it as “Beetles”, and then months later was bored and decided to research what it was and figured out the genus, I wouldn’t be able to update the identification on my own?

6 Likes

sometimes i think an observer’s own ID should be noted but not counted in the Community ID calculation. i think that might address different things that people in the Forum say irritate them.

4 Likes

Yes, it does. First, you might hope that “months later” someone else had already provided an ID. Second, I suspect that any measure like this is likely to have some downsides. For me, this one would be more than compensated for by the reduction in ‘automatic’ agreements.

1 Like

Which sounds similar to the idea mentioned above of requiring 3 IDs for RG, where one can be the observer’s ID.

1 Like

no, i’m not advocating 3 IDs for RG. that would not be the same as what i was talking about.

1 Like

I understand the distinction, which is why I used the word ‘similar’. The practical effect of removing an observer’s ID from the Community ID is to require 2 additional IDs, which is similar in outcome to allowing the observer’s ID and requiring a total of 3 IDs. In fact, I’d probably be OK with either of these approaches, or any others that have the same effect.

I will only agree with expert IDs if I can justify it, generally. If an ID is posted of some weird obscure species I can’t even find photos of, I’d rather look for literature to support confirming it. That said, when I know the identifier has very good expertise in a subject, I may be inclined to just go along with it.

3 Likes

This is what I call “adding weight” to an identifiers ID. I will only do it if I have developed a degree of confidence in their IDs, and as long as there is no contention over the ID. I am “around” to alter my ID (or even delete it) if the need develops.

I started in iNat just like everyone else, confirming the IDs of experts, a) because it seemed polite, and b) I was keen to get obs to RG. Nowadays I am far less concerned about whether they get to RG or not, and I am far less concerned about “fixing” wrong IDs from others. I think I have come to understand the concept of “Community ID”, but it took time! Hence I think 3 months of not having an Agree button (anywhere), and/or not having IDs made by new accounts (whether time or # obs/IDs based) count toward CID would help tremendously

I could see this being pretty discouraging to an eager new expert member who immediately uploads a bunch of accumulated observations in their area of expertise, only to find that their IDs are counting as zero against other random, non-expert IDs that show up on some of their observations. I’m a patient guy, but don’t think I would have the patience to wait 3 months for something like that to resolve.

I do definitely support more restriction on availability of the Agree buttons, however.

9 Likes

Yeah, the first time I suggested this I also added the possibility of curators “releasing” experts, or even non-expert that have shown that they “get it” as far as how CID and the Agree button should be used, but then it gets pretty discouraging to have to repeat yourself… Maybe I should have just put a link to one of the other times I suggested it.

1 Like