There is a user who does tons of identifications and uses an external AI for that (Merlin bird ID). Now, I do appreciate his honesty, he’s acknowledging it in the comment, but I don’t think this is right, for several reasons:
The current research grade system is based on two independent experts to identify an observation. This behaviour undermines and weakens this good system. If e.g. two people copied their ID from Merlin, it’s not independent any more, the ID process becomes a farce.
The AI is oftentimes wrong. much more than experts. Many times people send me recordings of “rare” species, as AI identified for them, and it’s a common Blackbird.
Person is always self-aware and knows when he/she’s unsure, AI not so much. When it somehow reports it’s level of certainty, it’s many times hard to interpret for us.
Please, make some steps to prevent people from doing this. E.g. post a guideline that we can link and post to people, and perhaps put some notification above the ID field, like “don’t post AI id’s as your own”. Let each person see it at least once, or press button “understood” or something.
There’s a guy in my county who uploads eBird checklists based on what Merlin heard. We’ve all had to just accept that if he uploaded it, don’t chase it and assume it’s wrong.
I don’t see the problem here. People are free to make identifications using any method they wish. If you think the ID is incorrect, just use your own ID to counteract it.
@tomas175 can I clarify? I think it is fine to upload your own observations with an app-derived ID like this (iNat has it’s own CV after all). But I think you are saying that this person is adding confirming ID’s to other people’s observations in this way… is that right?
I would certainly agree that that is inappropriate. Not even iNat’s own CV should be used in that way.
A confirming/correcting ID does not need to be made by an ‘expert’ (however defined), but it does need to be independent and human.
As @thunderhead said, iNat IDs are not restricted to experts. However, they should be restricted to the knowledge of actual humans.
This isn’t exactly accurate. The rhetoric from staff and iNat users on the forum has always been “the CV is not to be used beyond what you yourself can verify.” The same would apply to Merlin.
That being said, this is still the correct course of action:
And add a polite note explaining that IDs on iNat should be made based on the user’s own knowledge.
I have used Merlin while doing audio IDs. I listen to the recording and decide what it sounds like to me. If Merlin agrees, I feel confident in the ID and if it doesn’t, I either back off or do more research.
I do see a problem if the original observer posted an ID based solely on Merlin, not their own determination, and an identifier did the same. That is not legitimate verification. But I don’t think that is happening frequently.
I do a similar thing, but with BirdNET, sometimes. It helps me feel more confident in what I’m hearing if I’m a bit unsure.
What OP is describing sounds more like someone doing it blindly, though, and that is definitely more borderline. “IDs on iNat should be made based on the user’s own knowledge” gets a little fuzzy when it comes to making use of external resources – a literal interpretation would mean that keying out a species using a key published in someone else’s paper is improper.
That’s obviously not what’s meant, but is using Merlin all that different from manually keying something out using a not-perfectly-reliable key? It’s almost like an automated key, if that makes sense. It’s certainly less reliable than keys usually are, but I feel like drawing a line here would be subjective and somewhat arbitrary… but then again, there is precedent with it already being heavily discouraged to add corroborating IDs based purely on the iNat viz ID model without actually having any clue yourself.
I am pretty much of the same mind as Janetwright… I first check to see if mockingbird or other really good mimic is on the list of birds merlin has identified at the moment. A couple of days ago merlin identified a Caspian tern in my area. Certainly a possibility but because there were a couple of mockingbirds and brown thrashers raising hell at the same time I tossed the recording. I then listen to the recording and see how it matches up. Then does it make sense for it to be that species. Because of the numerous uncertainties of an Id based on the recordings I upload very few to Inat.
Correct. I worded that poorly. Rather “IDs on iNat should be based on the user’s own ability to verify.”
Yes, because the process behind the ID is completely unknown. A person using a key knows each diagnostic feature that was used by the key to make the determination.
I generally agree that a person making the original ID of their own observation using the iNat CV or Merlin or similar to make a reasonable guess is ok.
However, I don’t think it’s ok for an identifier to make an uncritical ID based solely on what Merlin (or other AI type identification service) says a sound or picture is. The person is not using their expertise or knowledge in this process. The only expertise would be knowing how to upload a sound/pic to Merlin or other tool - not making the ID itself. They would not be able to explain anything relevant about their ID other than “That is what Merlin said it was”.
If this process were taken to the extreme, iNat would just become a positive feedback loop of self-confirming machine learning based IDs. Obviously, I don’t think that has happened yet, but based on some of the CV-influenced ID issues we already see on iNat, it’s not an impossibility in some situations.
I would also note that this issue isn’t restricted to Merlin. There have been instances of other users adding large quantities of IDs based on submitting photos to other services like Pl@ntNet which I think runs into the same issue.
That said, I think there are legitimate ways to use these tools much as one might a field guide. People can use the iNat CV to get a list of five likely species, look into each of those, and then make an ID-based on what they find. In this case the CV output is being used critically to make an informed decision. The user would be able to explain their ID process meaningfully. In practice, distinguishing valid and problematic usage of AI-type tools can be quite difficult though.
A good initial approach might be to start a polite conversation with a user about their process. But disagreeing IDs are always fair game if one thinks that an ID isn’t supported by evidence.
I very often use Merlin to ID bird calls and then upload the recording from the app to iNat. Merlin is a tool, just like the CV here on iNat is a tool. People upload their photos, take the first CV suggestion and move on, to me it’s the same. I am not saying that’s the proper way to use either tool, far from it. just pointing out that they are both using AI for identification.
When I use Merlin I use it to get a good start on IDing what I’m hearing. I ask if the bird is in the appropriate habitat, location, time of year, etc. If I have never heard the bird before I listen to other calls and song recording to see if its a good fit, and if I am only identifying it based solely on Merlin’s ID I will state that’s what I’m doing in the notes with a disclaimer that I’m not sure at all. I usually only do this for birds I’ve never observed before or something uncommon. But Merlin is an excellent tool when I’m having a brain fart and know I know a bird but can’t think of the name, or to confirm what I suspect. I often need a refresher every year when some of the bird species come back, this year it was Black and White Warbler, I was hearing it and knew I knew it but couldn’t put my finger on it. Once Merlin ID’d it I was back in business. So I think it just comes down to teaching people how to use these tools and best use practices.
I sometimes post with a note that indicates that Merlin suggests something, but I would never ID another person’s observation based on Merlin. As for my own observations, I have great respect for the birders on here, and if someone suggests Merlin is not correct, I do not adhere to the Merlin ID. But, I pay a great deal of attention to my observations and update IDs even years the fact if I learn something new that suggests the original ID was incorrect.
It sounds like there is general consensus that using an AI to ID someone else’s observation without exercising your own knowledge and discretion is a bad idea. Now we need a guideline that says this. Otherwise we can share our opinions all day, but it won’t make any difference in cases such as this one.
Thanks for flagging this issue. This sort of behavior will very quickly snowball out of control without more direct guidance from the iNaturalist staff.
I have encountered a wildflower-identifier recently who relies totally on third-party computer vision for his identifications. When I ask him to support his incorrect IDs with identifying marks (e.g. “Hi there - What makes you think it’s A? It has the following features of B (…) and I can find no features of A.”) he either ignores the question or tersely states that it’s A because the third party CV says so. I looked up his name and it matched an IT professional who was involved in a major scandal at a school a few years ago (involving the electronic mass-fabrication of personal records).
I’m glad that he’s honest about where his strange IDs are coming from, of course, but it will only take a few more “identifiers” like him to override the human IDs, turning community IDs into useless AI echo-chambers. And, according to iNat, he’s not necessarily doing anything wrong.
Unlike with birds, plant observations often have very few eyeballs and a single rogue identifier can drastically alter the community IDs of whole species.
iNat does have a pretty clear definition of what an ID should be:
An identification confirms that you can confidently identify it yourself compared to any possible lookalikes. Please do not simply “Agree” with an ID that someone else has made without confirming that you understand how to identify that taxon and can rule out similar taxa. If you agree with the ID without actually knowing the taxon, it may reach Research Grade erroneously, which is detrimental to iNaturalist’s overall data quality.
While it doesn’t discuss AI suggestions specifically, it is clear that blindly following AI suggestions falls outside the bounds of what an ID should be.
Nah. This needs to explicitly rule out the use of algorithmic identifications. Otherwise, AI-identifiers have a very easy answer to the question this guideline poses: Do “you understand how to identify that taxon”? – Their honest, good-faith answer: Yes – I used an algorithm to do so! (and then I did the same thing thousands of times!)
I agree that it would be best to add a specific line addressing AI suggestions, but even as is, blindly following AI suggestions clearly falls outside of “confirming that you understand how to identify that taxon and can rule out similar taxa.”