New Computer Vision model released!

Hey all, the first update to the Computer Vision model since March of 2020 has been released! The iNaturalist website (and all network member sites) are using it as well as the Android and iOS iNaturalist apps. Seek, however, is still running on the older model as we still need to compress this new model so it will fit within the Seek app.

You can read more details on the iNaturalist blog and geek out at @alex’s more technical forum thread here.

Also, the About section of every species taxon page will let you know whether that species is included in the model:

43 Likes

Awesome! I’m assuming that this change also applies to the Computer Vision Demo… but just to be sure, does it?

There is also a new
Pending -
The current Computer Vision Model does not know about this taxon, so while it might be included in automated suggestions with the “Nearby” label, it will not have the “Visually Similar” label. While the requirements for model inclusion change with each model, generally inclusion is based on number of observations, so to increase the chance of this taxon getting included during the next model training, add or identify more observations of this taxon.

https://www.inaturalist.org/taxa/717367-Gladiolus-merianellus

1 Like

Yes the CV Demo page uses the same model as the website and the mobile apps. As mentioned in the blog post, the only exception is the on-device CV model that Seek uses, which we plan to update in the near future.

2 Likes

The taxon Osmunda spectabilis, the American royal fern, also has the “pending” label, and sure enough, it does not appear in CV suggestions. Yet there are over 8000 research grade observations for it. Why is this?

2 Likes

Presumably because people have identified those observations as such? One does not need the CV to identify something.

1 Like

I suspect that’s due to the (relatively) recent split: https://www.inaturalist.org/taxon_changes/76311 Osmunda regalis is in the model: https://www.inaturalist.org/taxa/82303-Osmunda-regalis

3 Likes

Hmm, so one downside of using an output as an input in a taxon split like that is that Osmunda regalis sensu stricto computer vision suggestions are likely based in part on many photos of Osmunda spectabilis?

5 Likes

Ah, I had a lack of context.

I thought it might be something like that. It is suggesting O. regalis if I “include suggestions not seen nearby.”

This isn’t a big problem for me, and I don’t really need to pursue it. I’ll just input O. spectabilis manually for the time being. Thanks for looking into it.

1 Like

Is anyone else noting the suggestions of Anseriformes being really bad now? They were reliable before but right now the CV is having trouble recognizing Mallard and Canada Goose.

1 Like

Please include specific examples, it’s much easier to investigate a problem if we can try to replicate it on our end.

2 Likes

Mallards seem to be usually recognised at Mallard x American Black Duck hybrids, and Canada Geese as Canada Goose x Greylag Goose hybrids.

1 Like

Yeah, I’m concerned about vision accuracy on avian hybrids & related species. It’s turning out to be a hard problem for the vision system. We don’t train on subspecies, and it may be that we shouldn’t train on any infraspecific taxa. I’m planning to do experiments next month to decide whether we should exclude avian hybrids in future models, or otherwise treat them differently.

The background is that previous versions of the model had far fewer avian hybrid photos to train on. We had a large growth in the number of avian hybrid identifications in the past year and for the first time, our (capped) training data had as many Mallard x American Black Duck photos as Mallard photos and American Black Duck photos. We didn’t exclude them from the training dataset because it wasn’t a known problem, but obviously I’m re-evaluating that now.

9 Likes

Is there any way certain taxa could be trained above the genus level? Many (most?) ciliates (to say nothing of other microbes) are not possible to identify to species with home equipment. Even genus can be quite challenging at times, sometimes impossible without complex staining techniques using chemicals not available to the amateur.

I don’t think that’s possible, since taxa above genus level have many genera under them, which might have completely different habitus. The AI still will suggest families and the like if it finds multiple genera that look similar to the observation.

Genus level is exactly the level I would like to see (this is the typical goal of microbial ID). But since I see no computer vision note on the genus pages, I assume they are not being trained at that level.

e.g. https://www.inaturalist.org/taxa/1094458-Opercularia

“We’ve also released a new feature for taxon pages on the website which allows you to see which taxa are included in the model. This badge only appears on species pages, not pages of genera, families, etc.”

That doesn’t address my question. So to put it clearly: can it be trained at the genus level? If so, how do we know?

yes it can. To quote Alex who runs it, “If enough photos are present at the genus level, but not enough photos for any of the descendent species, then the genus will be placed in the training set”

I posted the quote in my previous reply to explain why the tag symbol doesn’t show up on the genus pages

no idea

2 Likes