CV suggestions no longer accurate after City Nature Challenge

@sbrobeson, @tiwane, if you want LOTS of incorrect IDs in iNat, then go for it. This photo also contains leaves of Geranium dissectum, Geranium purpureum, blackberry, and others. But the predominant and most common are the Geraniums with pink flowers. This is NOT Geranium molle. It’s some other Geranium species that is not even on the list. Perhaps G. rotundifolium. Anyway, the CV model’s suggestions are all wrong.

Sorry for that last snarky comment. I don’t feel like you’re understanding me. I’ve given you 50 examples where the CV model does not give the correct ID. There are hundreds more like this. I’m telling you that this is unusual. I’m telling you that until recently the model was much more accurate in its suggestions. When the model gives incorrect IDs, it perpetuates and amplifies the problem. It doesn’t help for you to excuse each mistake the model makes.

The CV is very heavily trained on photos where the focal organism is dead centre in the photos. The organism in the very middle of the photo here is the thistle. I understand that the geranium suggestions may not be great in the uncropped photo but that is not what the CV “thinks” is the point of the observation. The simple cropping test that @thebeachcomber just did has better suggestions – again because the centre of the photo has changed.
I earlier in this thread strongly agreed with you (if I’m understanding the point here) that it’s likely that the CNC has changed the “Expected Nearby” behaviour. However, I stand by my remark that the particular example Tony highlighted is not a good example of that downgraded CV behaviour. It is working as intended given the particularities of that photo. I suspect there are many better examples where the behaviour is indeed anomalous, I just haven’t gone through all of yours yet (or all the ones in areas affected by CNC projects with a high error or malicious identification rate). I suspect that the moss example I gave may be part of the same phenomenon, although part of that one could be the cumulative effect of last year’s CNC as well…

3 Likes

If they get corrected at a later date, is the damage (semi)permanent? Or will the CV be corrected the next time it’s retrained?

In addition to the above problems, I think the “Identify” module and “Edit Observation” module are calling different code to display suggested IDs.

Here’s an example and steps to reproduce:

  1. View the following observation by entering the URL into a browser: https://www.inaturalist.org/observations/276044768

  2. Click on the “Suggest an Identification” tab and then click in the “Species Name” box. You’ll see the following list:

  3. Now view the same observation in the “Identify” module using this link:
    https://www.inaturalist.org/observations/identify?place_id=829&taxon_id=55978

  4. This will display a page full of observations. Look for the same observation as in Step 1 and select it. (There might be a more specific method to display this observation on the Identify page, but I don’t know how.)

  5. Select the “Suggestions” tab. You’ll see the following suggested IDs. Note that they are different from Step 2 above.

I’ve noticed that the suggested IDs on the “Edit Observation” page are much more accurate than those suggested on the “Identify” page.

By the way, the expected ID for the observation in this example is Malva multiflora (Cretan Mallow).

I’ve noticed when uploading observations that sometimes the CV suggestions will differ across multiple pictures of an observation. Not sure if there has been an increase in this or not, but it may offer a clue as to what kinds of images are “confusing” to the CV. E.g. it may suggest the wrong species on a flower, but still get it right on pictures of the leaves and fruits (recently happened to me here). I interpret this as many people focusing on getting close-up pictures of flowers (ignoring the rest of the plant) and misidentifying those. It’s fairly straight forward to check the CV suggestion on all pictures before combining them when uploading a batch, but doing this as an identifier is more challenging (related feature request).

2 Likes

CV is updated about once a month. If the batch of obs is sorted, next time it will be corrected.
https://www.inaturalist.org/blog/110120-new-computer-vision-model-2-21-with-over-1-000-new-taxa

This one is trained off data exported on February 16, 2025

1 Like

I know almost nothing about mosses, but there are even some angiosperms among the observations of this one.

I am surprised to see no G. pusillum suggested. In Czechia many photos of G. pusillum are CV suggested as G. molle, despite the latter being much rarer (NT with a negative trend, actually) and the former being a noxious weed. There are still many observations ID’d as G. molle from CNC that someone has to go through and correct many of them.

In the example you’ve given, the “Source” selected is “Observations”, so it will just show you species from the selected genus that are found in San Diego County (commonest first). Computer Vision is not involved at all. If you want to use Computer Vision, you need to change “Source” to “Visually Similar”.

5 Likes

Thanks for the explanation.

I suppose it may be be useful then to postpone the next update by a little bit to give IDers a bit more time to clean up the issues caused by the CNC so they don’t get added to the new model

Thank you so much for letting me know this. I’m new to using the Identify feature. Before I only used the Edit Observation feature.

1 Like

Since the date when a sp was added to CV is only in the links in these blog posts I add a comment to my own obs of ‘new to CV this month

Yes, if anything the vast majority of recent CV-driven IDs of this name are completely wrong… some students don’t understand the difference between mosses and small angiosperms. I still need to go through them but would welcome any help from other identifiers.

1 Like

Put a link on the Identifriday thread? (Checked ours for Cape Peninsula - all mosses)