Does iNaturalist log what the AI guess is for each record?

[kjhung]

Hello all,
Just wondering, does iNaturalist currently log what the computer-vision AI thinks the image is, at the moment of uploading, before the user / community weighs in with our human-vision IDs? If so, can we look at these data? I think there’s some cools stories here. If not, please let me know, and I’ll put this in a feature request thread.
Thank you! :)
James

[pisum]

i don’t think this already exists as its own feature request. however, there’s quite a bit of discussion about this as an alternative suggestion in a feature request thread that proposes something similar: https://forum.inaturalist.org/t/automatic-inat-suggestion-for-unknown-observations-that-reach-a-certain-age/4242/18.

[ahospers]

@pisum

y exists as its own feature request. however, there’s quite a bit of discussion about this as an alternative suggestion in a feature request thread that proposes something similar:

I was also wondering about the results, achievements, improvements of the differnt models as tere is a lot of rumour of using this ai models at observation.org.
For the testrun they have a distribuiton set

[ahospers]

But i think this is mainly for comparising models ?

[ahospers]

But i do agree there are cool stories waiting but i think the best results and reports are by comparing models with using the same training data

I have no idea where this image is from

The actual image i wanted to post i can not find, probably it is from another source (Naturalis, not iNaturalist)

[kjhung]

Thanks for the insight, both of you!
@pisum, given the existing discussion in that feature request thread you mentioned, do you think I should still make another feature request? Or would that be redundant?

[pisum]

honestly? i wouldn’t bother because it’s unlikely to be done, i think.

but there’s no harm in writing up a feature request. worst case, the mods will say that the existing feature request covers things already.

[kjhung]

@pisum

but there’s no harm in writing up a feature request. worst case, the mods will say that the existing feature request covers things already.

Gotcha, thank you!! I will do just that.

[ahospers]

I would like to know the difference/improvement in the different models for the different ordes (plants, fungi, birds). Somewhere there was a nice example for it but i can not find it. How do you know that a new model is better than the old one ?

https://forum.inaturalist.org/t/new-computer-vision-model-released-august-2022/34588/20?u=ahospers

[dianastuder]

https://www.inaturalist.org/posts/search?utf8=✓&q=new+computer+vision+model&post[parent_type]=Site&post[parent_id]=1&commit=Search

Maybe in one of iNat’s blog posts?

[optilete]

@ahospers

How do you know that a new model is better than the old one ?

Here is a standard blog for a new CV release.
https://www.inaturalist.org/blog/82291-a-new-computer-vision-model-v2-5-including-1-454-new-taxa

If -overall- a new computer vision model performs more bad then the old model, I should expect that the old model would not be replaced… So a new model in PRD should always be better?

[ahospers]

Well the actual graph i was probably from AINature.eu and showed the performance of each species against the number of available photos and the number of observations.
I showed there was a relation between the accuratenes/prediction of the model and the number of the photos.
I thought it was from a powerpoint for Belgium

https://forum.inaturalist.org/t/new-computer-vision-model-released-august-2022/34588/20?u=ahospers

[optilete]

I found this:
https://forum.inaturalist.org/t/opinion-on-the-computer-vision-icon/40910/64

@optilete

Compared to existing published works, our method
consistently achieves SOTA results on multiple datasets,
specifically 76.81%, 83.67%, and 91.39% top-1 accuracy
on iNaturalist 2017, 2018, and 2021, respectively

but I am not sure if this means that a new model in a new year means a better top-1 accuracy.

[jasonhernandez74]

I always try to add my input before the CV can make a guess.

[ahospers]

What is SOTA?
https://www.inaturalist.org/blog/72693-a-new-computer-vision-model-including-1-383-new-taxa-in-40-days showed
“…Our main goal has been to keep accuracy about the same as the model gets bigger more complex with more species. This is a few models behind, but the trend has continued…”

nfortunately there’s no tag, but there is a link buried in each post linking back to the previous post. Here’s all of them going back to March 2020:
https://www.inaturalist.org/blog/31806-a-new-vision-model
https://www.inaturalist.org/posts/54236-new-computer-vision-model
https://www.inaturalist.org/blog/63931-the-latest-computer-vision-model-updates
https://www.inaturalist.org/blog/69193-new-computer-vision-model
https://www.inaturalist.org/blog/69958-a-new-computer-vision-model-including-4-717-new-taxa
https://www.inaturalist.org/blog/71290-a-new-computer-vision-model-including-1-368-new-taxa-in-37-days
https://www.inaturalist.org/blog/72693-a-new-computer-vision-model-including-1-383-new-taxa-in-40-days
https://www.inaturalist.org/posts/72692-a-new-computer-vision-model-including-1-403-new-taxa-in-32-days
https://www.inaturalist.org/blog/74424-a-new-computer-vision-model-including-1-465-new-taxa-in-40-days
https://www.inaturalist.org/posts/75633-a-new-computer-vision-model-v2-1-including-1-770-new-taxa

[rupertclayton]

@ahospers

What is SOTA?

Probably a jargon acronym for “state of the art”. So, “…dynamic MLP consistently achieves SOTA results” is a way of saying “we didn’t win but we didn’t lose either”.

[cthawley]

That is also what I think, but my interpretation of SOTA there is “we’re doing just as good as or better than other models are doing” even though the IDs aren’t perfect, but YMMV.