Quality control checks?

I searched topics when deciding to post this. I would like to pick back up on the last comment of a closed topic…

Forum Moderator

Nov '19

That will be an issue any time you are near the transition zone between two biomes, and probably not a situation that iNat can be taught to recognize.

Most of my observations are from the north side of Alachua County, Florida. This is an area that has an overlap of species from Central Florida and North Florida, which are two very different biomes for many species due to the climate differences.

I haven’t been on iNat very long, but I’m already starting to see a pattern of identifications by observers well south or north of here jumping to identify a species per what they have seen in their area, even though there may be clear differences in appearance possibly due to differences in environmental conditions. I will use Erynnis baptisiae as an example. It is well documented that this species is expanding its territory, and appears differently in different locations. Unfortunately, any Erynnis observed in my area seems to get an automatic identification of Erynnis horatius even when there is an obvious difference in size and markings from what is ‘typical’.

How does iNat approach quality control for this sort of situation? I know that there is competition in any sort of human endeavor, but when a species falls through the cracks of our understanding of how it is changing due to habitat inavailability, climate, etc. because of folks jumping the gun and placing it in the incorrect taxon, or never getting it to RG because of identifiers outside of the observed area seeing it through a narrow lens, it seems like a topic that warrants attention.

How are these final ‘Research Grade’ identifications screened?

Thanks :)

2 Likes

To clarify, by “automatic identification” do you mean the top ID suggestion provided by the site’s computer vision?

No. Sorry for the confusion. I meant that those who subscribe to a taxon, and get first notifications of its posting, may want to take credit for the identification by quickly selecting a species when it may warrant further consideration.

Ah so you mean its actual users who provide the ID?

Hmmm, if that is the case then I’m not so sure. The most obvious thing I can think of is (at least in the desktop version) when you provide and ID that is different, there will be a popup with two options essentially asking how accurate you think the observation can be ID’ed to, so I guess that plays somewhat of a role in quality control. Not sure of other measures that are put up.

Of course, iNat is filled with people with various levels of expertise. It is going to take time for any research updates regarding the taxonomy, ecology etc. of a certain species to trickle down to the layman level. There is also an element of human psychology in that too; you essentially highlighted my thoughts regarding this in your final paragraph.

3 Likes

Soo… Hypothetically speaking… If a species incorrectly makes ‘Research Grade’ repeatedly due to mistaken identifications by overly enthusiastic laymen, and those misidentified photos go into the image gallery of the species, wouldn’t that taint future identifications since the image gallery is the resource laymen use for identifications?

5 Likes

Yes, this is a problem I have discovered with other taxa as well. (Or a learning opportunity, haha).

Most of the overzealous/overconfident identifiers I have encountered are perfectly fine with being corrected after being sent a message, especially with links to a source. Many will then review their identifications and bump them back up to genus.

The problem gets really bad once all these overconfident identifications by humans end up entraining the computer vision algorithm to also become overconfident.

12 Likes

I guess what I am really asking is if the image gallery is screened periodically by scientists skilled in correctly identifying specific species.

Exactly.

The final “research grade” observations are not systematically screened by a panel of experts. Some people with lots of expertise, professional or not, do make a hobby out of screening observations for their taxa of interest.

If you see a recurring error, and you have a few minutes/hours/days (depending on the scope of the problem) there are ways for you to help us correct the identifications. You can add corrected identifications yourself, or “tag in” other iNaturalist users with “@” followed by their username. Some people make journal posts with helpful tips for common identification errors.

8 Likes

I appreciate the guidance. Thank you :)

1 Like

One thing I’ve started to do more … when I’m confident that the submitter’s ID is incorrect but I’m maybe only 90-95% sure of my ID, I’ll add a comment indicating my suggested ID rather than provide the ID. I’ve seen too many cases where the submitter will automatically agree with the suggested ID of another – who might not be fully confident — and that leads to erroneous RG record. I still think 3 agreeing IDs are best for RG.

7 Likes

Yes, I’ve noticed that there is really not enough communication, and way too much just scrolling through dashboard updates and clicking on IDs. Dialogue is extremely valuable. It’s a shame that our culture has become so automated and distant. I think it’s our responsibility to not let that be at the expense of understanding the natural world.

1 Like

iNat is first to encourage us ordinary people to notice nature.
The scientific data is a secondary benefit. Which makes life interesting here.
(I naturalist, not I scientist)

3 Likes

Precisely.

I’m a Synechist. Engaging with a ‘Community of Inquirers’ is the avenue to truth. Again, dialogue. That’s why I posted this topic. :) It was not a complaint. It was just meant to open up discussion, and perhaps bring it more to the forefront of the community mind.

1 Like

That is why I follow my notifications. It can be fascinating to watch the discussions swirl, like the tide rolling in, and out.

3 Likes

In a way, yes! Here is one way: https://forum.inaturalist.org/t/computer-vision-clean-up-wiki/7281

Other ways are just by people interested in a taxon or species methodically going through records to check for “false positives”- (like @ddennism would for certain goldenrod galls heh. -Thank you for your service!).

3 Likes

I’m not one of the methodical QC reviewers, but I will occasionally dive into records of particular interest from my area and double-check IDs for RG records as well as check the unreviewed records. One way to catch some of these is to look at the distribution maps for your favorite species and focus on the outlier records … fixing those, to me, is important since they can lead to a cascade of further erroneous records.

There are simply way too many records to have a comprehensive review process. So it does take time to correct erroneous records which can languish for a while, sometimes for years.

8 Likes

The outlier idea is great! I do that randomly if I happen to notice it on a species’ map, whether or not I know something about the species. Then I end up learning something about the species by comparing the “probably is” vs “probably is not” record pools picked from the map areas.

(BTW Brefeldia maxima, good example- from that cleanup page cited above- that is currently getting cleaned up after years yay.)

3 Likes

I also feel like two ID’s is probably too few to warrant Research Grade. It would be nice if someone could spot-check the accuracy of RG IDs for various taxons from time to time, so we have some data to go by, but who has the time for that?

1 Like

Lots of us do that. :)

5 Likes