Weak points of Computer Vision Model

I have seen several cases where things unrelated to the genera get id’ed as Vekunta, Bruchomorpha, Pyrilla, or Ugyops. are they other such genera? This will help iNat improve the CVM (Computer Vision Models) for these.

1 Like

For more info on Computer Vision there is an iNat blog post with each update CV 2.24 August 2025

And lots of earlier forum threads for you to read.

1 Like

I want identifiers to highlight the taxa that CVM misidentifies a lot so that iNat can improve those weak points based on your feedback.

1 Like

Yes. Read the many earlier posts first …

2 Likes

… because then you’ll maybe start to realise it’s not as simple as that or everything would have been magically fixed long ago.

1 Like

Like magic? The overnight fix to default the map to satellite view :heart_eyes:

Everything else … takes years. With good bits on the way.

1 Like

I’m not sure who or what you’re referring to when you say “iNat” in the previous sentence. Do you mean iNat staff or iNat users?

If the algorithm does poorly with respect to some taxon, that is almost always because the iNat users have done poorly (or not at all). It’s been said many times but the algorithm is only as good as the data fed into it. If you want the algorithm to improve, you have to improve the training data (more observations, better identifications, etc.).

It’s easy to find taxa for which the algorithm does poorly (no list needed). Pick one, and do your best to make it better. Don’t worry that there are too many. Every taxon you help to improve is a win.

12 Likes

Exactly, it just takes a lot of time. My pet genus Acrolophus was pretty bad when I started on iNat, and 42k IDs later the CV now gets them right most of the time.

15 Likes

this is a decent place to start reading about how people have thought about this problem:

https://forum.inaturalist.org/t/computer-vision-clean-up-wiki-2-0/40318

2 Likes

This sums the problem up nicely:
https://forum.inaturalist.org/t/north-american-sinea-id-and-the-sorcerers-apprentice-problem/68337

There are two issues with suggestions:

  • The app takes the visual similarity score of the CV as ID confidence.
  • Good observations are completely ignored because the CV is not trained on the parent.

I wish the CV would improve, too.

4 Likes

Heads-up that including specific examples is really helpful when bringing up questions, issues, etc. Like screenshots of what you’re seeing, or URLs of specific pages. Without those we’re all making a lot of assumptions about what you’re saying, and can often only speculate.

6 Likes

iNat cannot identify these genera accurately and keeps on misidentifying a plethora of unrelated things (e.g. moths) as them, so I would like to see these specific taxa improved in the the next CVM update.

Are you seeing this from the iPhone app, the Android app, or the website (on an observation page, on the web uploader, or on the Identify page…)?

I looked up the 4 genera you listed and all of them are insect genera with not many observations of any species in the genera. For 3 of the 4 genera, none of the species have enough observations to be included in the CV. Thankfully at least at genus level there are enough observations to be included.

1 Like

I just went through a bunch of them too, and a few things stood out. Vekunta and Bruchomorpha didn’t look too bad, Pyrilla had a couple dozen clear misidentifications, and I’m still looking through Ugyops. What stood out to me was:

  • 0% of the misidentifications had the genus in question actually suggested as a good match by the CV. Either the Vekunta/Bruchomorpha/Pyrilla/Ugyops genus did not appear on the CV suggestions at all, or the CV said “We’re not confident enough to make a suggestion” and the genus in question appeared under that warning as essentially a wild guess.
  • Nearly 100% of the misidentifications were of uncropped photos in which the insect took up only a tiny proportion of the photo, so the CV really had no chance of getting an ID right

I’d say the takeaways from this are:

  • Don’t use the CV to auto-fill an ID for an observation when the CV explicitly states “we’re not confident enough to supply an ID”
  • Crop your photos down to the organism in question if you want the CV to identify it

I don’t think there’s any problem with the CV model for these genera; there are just some newer users who don’t understand how to get an identifiable image or don’t notice the difference between a CV suggestion and the CV’s “guesses” when it’s stumped.

I know this has been suggested elsewhere, but my productive suggestion is that I would love to see the CV stop supplying guesses under the “We’re not confident” message. The only options under that message, in my opinion, should be “Plant”, Animal”, Fungus”, or “I don’t know”. Imagine if someone showed me a photo so baffling that I had no idea what I was looking at, and I responded with “I have no idea what that is… but maybe it’s a EURASIAN LYNX, SLIGHTLY-MUSICAL CONEHEAD KATYDID, or EUCALYPTUS TREE!!!1” It would have been better for me to stop at “I have no idea”, because the rest of that statement is just ridiculous and counterproductive. I’d love for the CV to have the same restraint.

14 Likes

Yes, or provide a really broad answer. If the top suggestions are lynx and katydid, suggest “Animals”. If they’re both insects, suggest “Insects”!
Feature request: In cases where the CV is not “pretty sure” of anything, offer a suggestion of a higher taxa - Feature Requests

The new Next app seems to be better at handling this aspect, while being worse in terms of suggesting species when it should suggest genus like the web CV does:

5 Likes

Website.

This would definitely solve a lot of issues that I have with the CV. I’d be perfectly happy sorting through a couple thousand Annelida, Oligochaeta IDs rather than having to stick disagreeing IDs on a couple thousand misidentified “Lumbricus terrestris” which get stuck at subclass or order anyway since not too many observers go back and change things. The CV is overconfident, even in its “not confident” IDs!

5 Likes

Here is an idea - What id CVM did not need updates but used AI to find similar observations and suggest what ever ID those observations have been given on average.

EG: A species is similar to one observation with ID Planthoppers, another with Sarimini, and one being Euroxenus vayssiersi. Now it may suggest Issidae or hemisphaeriinae accordingly. @tiwane - What do you think?

I’m not understanding how what you’re suggesting differs from the way the CV works? It’s constantly being updated to take new data into account as more observations are added, new IDs are given, etc, and that’s unavoidable if you want things to improve.

1 Like

That is very much what the CV is. No one goes in and “teaches” the CV manually how to ID anything- it’s just an image-matching AI trained on the observations that have already been ID’d. This is why if the CV is making weird suggestions, the most likely reason is that the observations it’s comparing to are misidentified. “what ever ID those observations have been given on average” is going to be wrong if “those observations” have been given the wrong ID. So the solution to consistent CV errors is usually not to tweak the CV itself, but rather to go in and disagree with all the misidentified stuff on iNat so the AI is being trained on correct information.

4 Likes