Please provide feedback for Seek beta on iOS if you can!

General
1

Hey everyone,

Seek by iNaturalist has, until now, used an old on-device computer vision model from several years ago. It has about 20k taxa in it and really doesn’t work well for many parts of the world outside of North America and Europe.

We’ve added CV model 2.13 to the latest beta release of Seek (the same version currently used in iNaturalist Next), which has about 80k species in it and allows it to recognize more species both in the regions where it was already working well in addition to other parts of the world.

However, having a much larger model means we’ve had to tweak the in-camera suggestions so that the app provides suggestions at a satisfying rate while also being accurate - it has many more taxa to try and go through, which complicates the process. (That’s about the extent of my understanding at a technical level)

We think we have it in a pretty good place now but need some feedback from the community, so if anyone has an iOS device and wants to test out this beta version, it would really help us understand if Seek’s working well and is satisfying to use for you all.

If you’re interested, you’ll have to download Apple’s Testflight app to your device, then open up this link on your device: https://testflight.apple.com/join/LM5oO3P7 Install the beta version, scan some organisms and let us know here or at help+seek@inaturalist.org what you think. Is it accurate? Does the camera work OK? Are you getting suggestions at a satisfying rate as you scan organisms?

Please keep in mind that beta versions may not work, may have bugs, or may cause issues with your current Seek observations. If you have a lot of precious Seek observations on your device, there’s a very slight possibility they get deleted. Unlikely but possible.

Thank you!

1 Like
2

Here are a couple of iPhone screenshots from this beta update, for those of you who are curious about Seek, but don’t use it:

IMG_4613
IMG_46131125×1935 241 KB

IMG_4614
IMG_46141125×2039 417 KB

1 Like
3

I’ve done some testing with my collection specimens and here’s what I’ve found for accuracy (Just to note - I’m in Missouri):

Testudines - I tried to scan my box turtle shell, but it was very keen on it being a tortoise. The closest it got was “Turtles and Tortoises” which is what it finally settled on.

Insecta

  • Magicicada tredecassini - I tried two of these, the first I only scanned the top, which shouldn’t have been able to be brought to species, which Seek agreed with leaving it only at “Periodical Cicadas.” The second I flashed the bottom, and it immediately said “Pharaoh Cicada” or Magicicada septendecim, which was incorrect. I don’t think the CV has caught on to differences in Magicicada yet.

  • Necrophila americana - Was very fast and correct

  • Melanactes - Went straight to species “Melanactes piceus.” Can’t verify if it was correct or not as my iNat observation is only at genus.

  • Tarpela micans - Fast and correct.

  • Eburia quadrigeminata - Fast and correct

  • Epicauta vittata - Fast, but can’t verify because my iNat obs are only at complex.

  • Neopyrochroa - Fast to species, but, again, can’t verify species, but genus is correct.

  • Cybister fimbriolatus - Wouldn’t go lower than the family “Predaceous Diving Beetles” which is probably a good thing because diving beetles seem difficult to identify (unless you’re mpintar of course).

  • Perothops muscidus - This one I know isn’t in the CV, so I wanted to see what it would do. It’s also a bit of a trick being a False Click Beetle. Seek went immediately to “Click Beetles” and wouldn’t budge. That makes this one the second incorrect one.

  • It got the next four correct very fast - Neoclytus mucronatus, Megalodacne heros, Glischrochilus quadrisignatus, and Penthe pimelia.

  • Erotylidae from the Philippines - I have one Erotylidae specimen from the Philippines that Seek was very keen on being Tenebrionidae instead, and settled on family level.

  • Random moth - Only said “Insects” after scanning it for a while.

(Maybe I should do more other insects when I test the app next, I did a lot of beetles this time…)

Plantae

I did two plant specimens I have on the same species to see if it could say the same species for both. They’re both Henbit Deadnettle, but one has white flowers instead of the usual purple. Seek got the purple one no problem, but only got as low as dicots for the white one.

Deer Skull - Wouldn’t say anything for it, just said “We weren’t able to identify this photo.”

Fungi

I scanned my Trametes aesculi specimen, and it was surprisingly accurate getting to genus Trametes.

Unionidae

It had a very hard time with Quadrula quadrula, it kept saying very quickly that it was Obliquaria reflexa.

Birds - it got all my bird photos correct very fast.

Overall, it was pretty accurate, only getting my weird curveballs wrong, and was pretty fast for most things. The camera was a little funky with smaller things though, it would focus just beneath the organism, and I’d constantly have to be moving the camera up a little to focus.

In the coming weeks I plan to do more testing, and especially more testing out in my woods to see how it does with more living things.

Edit - I forgot to add this initially, but some of the common names are outdated. The common name for Megalodacne heros on iNat as of six months ago is “Heroic Pleasing Fungus Beetle” while on Seek it still has the original one, “Pleasing Fungus Beetle.”

4 Likes
4

Thanks for doing this test, and thanks for posting your results!

AFAIK (and Tony can comment on this), the folks who use Seek have a lower threshold for ID specificity. For example, an iNatter (especially one who does a lot of IDs) might care a lot about getting to species or subspecies, species complex, section or subsection or cultivar for plants, etc.

Whereas Seek users might have a question like, “What is this thing with legs? Is it a spider?” Genus would be more than enough for a lot of them (I could be wrong about this).

1 Like
5

Heh, Seek is actually more focused on getting species-level IDs because observations aren’t saved in Seek (and won’t count towards badges and challenges) unless they are identified to species. So if people are using Seek for those reasons, getting to species makes it much better experience for them.

Thanks! Would you say that it did a poorer job focusing on close-up objects than your phone’s default camera app? That’s been my experience.

2 Likes
6

Yes, it did a poorer job focusing on close up objects than my regular phone camera.

7

We’re slowly rolling out a new public update to both Apple and Android that includes these changes plus some other big ones:

  • the geomodel is now being used when making suggestions if the app has permission to access your location. This should cut down on mistakes like Apis cerana being suggested for Apis melifera in the Americas.

  • Seek for Apple now uses the system-level photo picker for importing photos, which should fix this bug.

  • Seek no longer suggests SARSr-CoV at all, so it won’t randomly do so whenever the camera sees a desktop or something similar.

Becuase these are some big changes we’re rolling out the release over the course of a week in case there’s an unacceptable increase in crashes or other errors (although I haven’t seen any issues in my testing).

3 Likes