Why does CV give different suggestions on the mobile app than on the website?

It’s not a regular thing, and yes, I’ve done it both ways. Your example is different in another way as well (the example with the lily) as there you’re giving the CV two different options in a single image, the bee or the plant, and, as has been mentioned in other threads, there appears to be a bit of an animal bias in the CV system, likely because that’s what many people are taking photos of.

What’s perplexing in the observation I linked is that there is only one organism for the CV system to work with, and it gets it correct to the genus level when it suggests an ID at upload time. It’s afterward, once it already has a correct ID assigned that it goes off the rails and suggests something wildly incorrect (both browser based and app based), and doesn’t even have the initial suggestion listed as an option.

You’d expect that the CV system, looking at exactly the same image on the exact same observation as it did previously, would come up with the same ID suggestion.

Do you have multiple photos? Are they in the same order? CV only looks at the first one.

2 Likes

Not sure why there’s a difference, but just confirming that the most recent CV model is used by both web and apps: https://www.inaturalist.org/blog/63931-the-latest-computer-vision-model-updates

I reproduced this behavior using the following method:

  1. Find a copy of the original observation.
  2. Copy the photo to my phone (Android).
  3. Created an observation using: “Choose Image”
    https://www.inaturalist.org/observations/120821320
  4. Click on “What did you see?”: the top suggestion was E. caespitosa; I did not select this left the observation as unknown.
  5. Submit the observation.
  6. Still on the phone, open the observation and use “Suggest ID”; the top suggestion was still E. caespitosa. Choose it and save.
  7. Switch to the website: use “Upload” to create an observation with the same picture.
    https://www.inaturalist.org/observations/120821759
  8. While adding this ask for the suggestions: the top suggestion was E. californica. Leave the observation as unknown.
  9. Submit the observation.
  10. Still on the website, open the observation and “Suggest an Identification”; the top suggestion was still E. californica. Choose it and save.
  11. Still on the website, open up that first observation and “Suggest an Identification”, the top suggestion was now E. californica.
  12. Back to the phone, open up the second observation and “Suggest ID”, the top suggestion was still E. californica.

The next experiment I could try would be to take the second observation, withdraw the ID, and use the Android app to “Suggest ID” again and see if it is different.

I don’t have an iPhone but to be complete I would want to try the same experiment with one.

So I agree with @beachcomber that a previous ID might be influencing the suggestion, but when there is no previous ID the CV does give different answers depending on the interface used.

1 Like

Note: my intention is to delete these two experimental observations later this month.
I also changed these two observations to be casual.

so the odd one out seems to be the ID suggestion during upload on the app.

there are 2 possibilities i would look at:

  1. it’s possible that the suggestion during upload in the app doesn’t incorporate the location properly to boost the nearby suggestions. you can test to see whether there’s a difference here by clearing out or adding a location to see if there’s a difference. for example, in the web uploader, if you change the location (the coordinates, not just the description) of your poppy observation from Martinez to San Diego (or if you clear out the location altogether), you should see slightly different suggestions.
  2. it’s possible that the app is providing a slightly different version of the photo to be evaluated by the computer vision. something similar was happening here: https://forum.inaturalist.org/t/did-something-change-with-the-ai/20834/14. you could try resizing your photo, and see if that makes any difference in the suggestions you get.

Both of those ideas are good things for the developers to look at. I merely created a reproducible scenario which can be used by others to figure out. I don’t have the time or energy to go into the code to figure out why we get different behaviors.

Regarding the second idea, if that makes a difference then we have a usability problem. No typical person in the field is going to resize a photo taking using their phone app before submitting the photo to iNat.

1 Like

Oh, a correction. I see the Android app might resize the photo before uploading to iNat. But that did not make a difference. I took the resized photo uploaded via Android then submitted it using the website; the website still suggested E. californica.

when you’re uploading in the app, are you manually adding a location?

in the app, if i start with your photo and create an observation with no location, i get E. caespitosa as the the first choice. if i add a location, depending on the location i choose, the suggestions may change.

1 Like

No location or a location 20 miles away from the nearest E. caespitosa: it suggests E. caespitosa.
Pick a location such as Denmark: it suggests E. californica unless I “Include suggestions not seen nearby” in which case the top suggestion is E. caespitosa.

(Oh, and just found another one of these incorrect Tufted Poppy observations in San Francisco a few minutes ago.)

ok. when you did you earlier test, you noted that when creating the test observation via the app:

so then did you have no location input at the time? (i assume you are manually inputting the locations, since the locations on your test observations differ slightly.)

In the earlier tests I always had a location set up first. In the test scenarios I used a location a few hundred yards away from the original observation.

The only time I had no location was for an experiment done a few moments before my previous post. I did not save that observation.

hmmm… i’m having trouble reproducing your #4 and #6 then. if i choose a location close to Martinez on the app, i get E. california as the top suggestion. if i open up your existing app-submitted observation and try to get a vision suggestion, the top suggestion is E. california.

(if you’re still able to consistently reproduce #4, then maybe the app is using your current location instead of the location of the observation, and you’re currently located somewhere other than Martinez? if that’s the case, then switching on and off location services on your phone and getting suggestions for each case should produce different results.)

Are you using Android or iPhone?
I am using Android version 1.26.11 (541)

A hint the app is using the location specified: when I choose a spot in Denmark it gave me the choice “Include suggestions not seen nearby”. When I took that choice it gave E. caespitosa as the first choice.

me too.

i was thinking maybe the current location could somehow trigger some different behavior that’s not obvious. all i know is that i can’t i reproduce your #4 and #6 on the app. if a developer can’t reproduce it either, then it’ll be tough to troubleshoot.

i downloaded a medium version of the first photo in the observation (https://static.inaturalist.org/photos/204307836/medium.jpg) and then started a new observation in the web upload screen. then i added your coordinates (20.745279,107.069327, acc 32m), and asked for a vision suggestion.

here’s what i got:

so i think you’re right that on upload, the app may have suggested Primulina, but i don’t think it’s a mobile app issue specifically.

i don’t know how the iOS app works, but i know that the web uploader and web observation / identify screens make slightly different API requests to get the vision suggestions, and the behavior matches what you described:

for completeness, here’s what i see in the website when i ask for suggestions on the observation screen:

one thing that i’m noticing is that this particular observation’s suggestions come with a “we’re not confident enough” disclaimer, which i think could mean that maybe there are very small differences in the scores of the vision suggestions. i think that could mean that if the two different algorithms do something even slightly differently, like round to different precision, you could end up with very different results. or maybe if use slightly different-sized images, you might get 2 different results.

that’s just a guess of what might be happening, but this seems reproducible enough that a developer could debug this and see exactly what’s going on, if they were inclined to do so.

2 Likes

Can’t help with any of the logistics but this happens to me constantly. I upload pretty much exclusively on the app because I’m faster that way, and the site gives different suggestions almost every time. Not exaggerating. Usually it doesn’t matter because I just do something general or say what I think it is without the CV but it’s made me feel crazy at times. Glad I’m not going insane and somebody else has it happen.

1 Like

can you provide a couple of screenshots that show an example of the app and the website giving you different suggestions? are the suggestions actually different between app and website, or is it between upload and observation screens? like if you open up an existing observation in the app and the website right now, are you saying that it will give you different suggestions between app and website?

1 Like

I cannot do a test in another area effectively.
I’m in an area that iNat thinks is local for E. caespitosa. I can do a test from San Francisco and San Mateo counties, and there are no observations of E. caespitosa in those areas, but iNat still thinks E. caespitosa is there. The wrong id observation I saw this morning was in San Francisco.

When uploading an observation on the app, it will give me different suggestions than when I go back to that observation on the site, after it’s been uploaded. So, while uploading I’ll be given A, B and C as suggestions, then if I look at that same observation on the site, it’ll suggest something like B, D and E. It’ll have some of the same ones sometimes but usually they’ll both include different things from one another.

I’m not sure if it does that same thing when checking an observation already posted and then checking suggestions. For example, if I check an observation my neighbor made 2 months ago on the app, I don’t know if those suggestions would be different from the site’s suggestions. I just know that uploading on the app vs looking at that same observation on the site will yield two separate sets of suggestions.