Unless I’m misunderstanding, this doesn’t seem to be an example of the case you mentioned - where the model suggests “we’re pretty sure it’s in genus X” but then the top suggestion is “species Y” where Y is not a member of X. Thanks to the example you found, we now know that this can happen in cases with the seen nearby filter active, which is really helpful. I’m interested if it happens in other cases as well, since this could be a bug in the code or a problem in our algorithm.
1 Like
Toggle to seen nearby? None of the ‘not seen nearby’ IDs are right.
Unless it is a cultivated garden plant, I will use seen nearby as it helps to avoid Northern (wrong) suggestions.
Ah, gotcha, I misunderstood, thanks! So this is indeed the problem, where we say "we’re pretty sure it’s in genus X” but then the top suggestion is “species Y” where Y is not a member of X, but I think it’s the same problem you identified earlier.
I think the issue here is that we compute the “common ancestor” from the computer vision results, and that’s what we show in the “we’re pretty sure it’s in rank X.” We then use the common ancestor as a starting point to augment the vision results with geo frequency data. Ken-ichi explains it here, starting at the 16 minute mark: https://www.youtube.com/watch?v=xfbabznYFV0. As you can imagine, if the vision results are really wrong (as can happen), then we may end up with a really wrong common ancestor, and that throws the rest of the algorithm off. I think that’s what we’re seeing here.
We are working on improving our approach to incorporating geo frequency data with vision results, but it might take a while before we can launch that. In the meantime, I’ll see if there’s anything we can do in edge cases like this. Perhaps we could update the wording to say something like “it looks like genus X, but that is not typically seen nearby. here are our top suggestions that are seen nearby.”
6 Likes
Yes please for rewording.
The Santolina example needs a new line between pretty sure and suggestions.
Visual match to the right species, which is not seen nearby.
Then followed by the hopeful suggestions.
1 Like
@alex In the few years I’ve been on iNat I have seen CV improve for our Cape Town fynbos (many obs, many IDs) Which motivates me to work thru Unknowns for the Rest of Africa.
Can you tell us how many species you added for Africa with this model?
I can’t do this directly so I’ll have to ask for help from my co-workers to see if we can find an easy way to do it.
Part of what’s hard to figure out here is that many of the taxa that appear to have been added to the model are actually just taxonomic churn - taxon A was replaced by taxon B, or split into taxon C and D. I don’t think we have programmatic ways of processing the taxon change history, so it’ll require a ton of bookkeeping.
I’ll let you know if we can figure out a way to tease out the true number of new species.
3 Likes
I made 130 observations yesterday and I noted this was visible in 4 of the autosuggests.
As with @dianastuder’s example, bug only visible when toggled to “seen nearby”.
2 Likes
(However, the first suggestion was Amphineurus when I included suggestions not seen nearby.)
2 Likes
thanks, can you share the photo or an observation made from it?
Thanks!
Another example for you if needed @alex
Says it’s pretty sure it’s Crotalaria (which is correct), but neither species option is in Crotalaria. If I toggle off seen nearby, Crotalaria lanceolata is the top suggestion, which is correct
Observation: https://www.inaturalist.org/observations/112012459
2 Likes
I would be satisfied with the available raw data number with churned taxa. (Maybe superimposed on a world map so we could each see it for our chosen geography)
That would be a number you could easily compare across each new model.
I like to see patterns and trends, not needing to see a perfectly accurate number.
I’m an enthusiastic iNatter not a research scientist. We anyway have to deal with - sigh - they have changed the name - again.
1 Like
There seem to have been a few weird, way out of range suggestions but I can no longer reproduce this. One example is this observation from North Carolina where the original CV suggestion was for a species endemic to Australia, but this is now no longer showing up among the suggestions. Maybe a temporary glitch right after the new model was released?
1 Like
Gotcha. I know that in the past this was something we were interested in as well. I’ll see what I can do.
I might not get to it until next week, but it’s on my todo list.
1 Like
I’m curious if this affected the Identify section - when you click on the observation in the Identify window, and it pulls up that observation, sometimes I cannot remember spelling or such close enough to get to what I know something should be, and I used to be able to use the “suggestions” tab. It seems like in line with the new CV model, now it no longer makes logical suggestions. Like. At all. Plant photos get bird & fish suggestions, for example.
I should have taken a screenshot - I just went and did this one for example and at least it figured out ‘flying thing’ correctly, but still WAY wrong and this sort of issue wasn’t happening before.
Oh here we go, this one shows the issue well.
On any of these, if you click to the observation page, and then use the CV, it works fine.
You’re filtering by Observations, not “visually similar”
3 Likes
I didn’t change the filter; it used to automatically do visually similar then.
Could it be changed back to automatically default to visually similar, rather than default to observation?
It has always defaulted to observation for me. I think it is must be less computationally intensive to default that way; I have ok internet and it still slows the modal down a fair bit to have it tabbed to visually similar. I also suspect a lot of the people who use the modal are just using that tab as a sanity check to see what else has been seen in the county/region, which the visually similar tab can’t reliably do for species not in the CV.
2 Likes
Interesting, thanks. I don’t use it that often, so maybe i’m just seeing it worse now by coincidence.
Thanks everyone for solving the issue though, now I know where to click to get what I want! :)
1 Like