Write up on Computer Vision

[gancw1]

I remember reading a document on how Naturalist computer vision works i.e. number of RG observations required, frequency of data feeds etc but I can’t locate the article now.

Can someone point me to that article ?

[bouteloua]

There are these FAQs: https://www.inaturalist.org/pages/help#computer-vision

By frequency of data feeds, do you mean how often is the model updated?

[thebeachcomber]

Perhaps this paper has some of that info
http://openaccess.thecvf.com/content_cvpr_2018/papers/Van_Horn_The_INaturalist_Species_CVPR_2018_paper.pdf

[gancw1]

Thanks. I remember the theeshold was something like 20 RG observations. Has this been changed to 100 community id?

And yes I was wondering how frequest new dataset get refreshed ?

[gancw1]

thebeachcomber,
Thanks for the link

[tiwane]

There’s also this blog post: https://www.inaturalist.org/blog/25510-vision-model-updates

And our CV FAQs: https://www.inaturalist.org/pages/help

[leejones]

Thanks all for the useful information everyone! I see that there are a number of species that have fallen prey to issues such as the “overconfident computer vision” issue and have been trying to understand as many aspects as possible. In trying to understand more about iNat’s AI / computer vision all of this has been extremely helpful.

[bouteloua]

@gancw1

And yes I was wondering how frequest new dataset get refreshed ?

Just wanted to copy in this response here to answer your question:

@kueda

We’re currently for shooting for twice a year, but 2019 had a lot of interruptions in the form of experimenting with and training a model that could fit on a phone for Seek. Given the current way we do it, it can take months from start to finish, depending on how accurate we need / want it to be, how much QA / QC we do, problems we encounter during the process, etc. We’re shooting for at least twice a year, which we’re hoping is reasonable and gives us enough spare time to experiment with different stuff on the same hardware. It’s possible we’ll figure out a way to update the model without fully re-training it, which would allow us to add new taxa at a higher frequency. That’s one of those experiments we haven’t had time to do yet.

[upupa-epops]

From the help page:

Taxa included in the training set must have at least 100 photos, at least 50 of which must have a community ID. As more observations are added and more identifications made, additional taxa can be added to the computer vision suggestions. This means your observations and IDs work to improve the model!

Can any observation be used for training or just verifiable observations?
For example from my interpretation of that, a common cultivated species could have 50 observations, all marked as casual/not wild, with an average of 2 photos per observation, and would be included in the CV training set as long as someone other than the observer has made an identification on every observation. Is that accurate?

[bouteloua]

Some non-verifiable observations are included in the training:

@kueda

The vision system is trained on captive / cultivated records. However, our automated suggestions are based on vision results and nearby records, and the nearby records part of it is currently only using RG records, so if vision ranks Canary Island Pine highly, it might get knocked down by the legit lodgepole records in the Transverse Ranges.

Patrick tells me we haven’t run our accuracy tests without the RG requirement, so we’ll do that and see if it makes things better or worse.

Sort of tangential, but it’s also worth noting that our policy of assuming that cultivated plant obs don’t need more IDs means that our cultivated plant records (both images and occurences) probably have less-accurate identifications and are thus less useful for either purpose (probably why we put that RG requirement in there to begin with). I’m aware there are many who would prefer that we remove that part of our quality grade assessment, and it might help situations like this… or it might just make even more identifiers give up in the face of endless potted plants.

@kueda

Observations in the test set for vision and the test set for the whole system are drawn from Research Grade observations and observations that would be RG if they weren’t captive (RG+Captive for short).

[upupa-epops]

Ah, thank you! Sorry I didn’t see that thread.

[dianastuder]

Wouldn’t it be easier, if computer vision, cleared the potted plant backlog?

With that stated goal of encouraging ordinary people to notice nature, you will get lots of cultivated plants. Ordinary people aren’t field biologists, they have to make do with what they can see, in the cities.