Currently, the CV only suggests species for the first image in an observation. It would be useful to be able to get suggestions on any image in an observation. There are ways to achieve this in the current system, but they are cumbersome and only work on a user’s own observations, not those of other people. Intuitively I want this to happen when I scroll to the next image in the viewer, though there might be better ways to implement it.
I agree that would be useful. But shouldn’t the computer vision suggestions be on the whole set of photos anyway, it doesn’t make sense for the system to only look at one photo when suggesting an ID for the observation.
4 Likes
I’m no programmer, but that seems a lot harder to implement. The computer vision is not programmed to identify organisms, rather pictures of organisms. It seems unlikely it would be able to collate information from different pictures the way a human can (i.e. it can’t really think along the lines of the leaf is shaped this way, the flower is shaped that way, therefore…).
Being able to choose what picture to invoke the computer recognition on, though, would be invaluable.
10 Likes
I agree. Asking the CV to weight and combine its guesses for multiple pictures in an observation seems to cross the line from helpful tool to scientific judgment. Integrating those guesses and sorting them by overall plausibility is the job of human experts, at least for now.
3 Likes
Determining the common ancestor of the taxa recognized in different pictures from the same observation sounds like a straightforward proposition though, unless I’m missing something.
4 Likes
I agree, either that or the original image-by-image request should be straightforward. The crux, of course, will be whether the added computational load will be feasible.
1 Like
From my non-technical perspective: the ability to choose which photo to run the model on is probably doable, but use multiple photos at once to determine a suggestion is probably not. Or at least would not be doable in the near future.
6 Likes
I would have thought just process each image and take a majority vote, and default to the first image if indeterminite (although as noted above, a high computational load). No agreement between images might suggest going for a higher taxon, or less confident suggestion.
But being able to select an image would work fine on the website version. Combining that with the ability to draw a square/circle around the element that is the focus of the observation would be a bonus (that’s another feature request too I think).
2 Likes
I think @Megachile is requesting that for observations with more than one photo (on desktop), that the feature be added to switch CV suggestions from the first to another photo in the set when performing IDs on observations that are not ones own.
I think this would be best implemented in the ‘compare’ tool, so when performing identifications on PC it’d be more efficient for comparing different characteristics from each photo as you can already do on mobile.
I’d really appreciate the feature as well, and I suspect others who don’t use the forums would as well.
3 Likes
This is the workaround I had in mind in the OP as “cumbersome”
5 Likes
The other issue, is that this only works when there is 1 species in all pics.
Very, VERY often, users (especially new users) will upload pics of multiple species in 1 observation.
I haven’t seen the CV go to “Life” often, so it would probably come up with something really off the wall to try and fit all the pics.
Though I suppose one could argue that this is a moot point: the end result (a suggestion that doesn’t apply to all the species in the photos) is ultimately no different now when it only looks at 1 photo.
1 Like
I was just requesting the same thing. I think using all images would be cumbersome to implement, but just using the image a user scrolls to (as in panel 4 of the attached image) should be easy.
2 Likes
That’s how it works in the android app.
Not in iPhone or the website, though.
4 Likes
I think it would really be quite straightfoward to combine the CV results from all of the photos into a single result. For each image, the CV model produces something that looks like a probability distribution for all of the possible IDs. The natural way to combine these would be to add them all together and normalize the result. I think this is mathematically correct, assuming that you want each photo to influence the result equally.
2 Likes
Most people don’t post multiple photos. But, you could also make it an option, whether to base the ID recommendations on a single photo or on all of them.
Most people who post more than 10 observations likely do post multiple photos.
3 Likes
You and I do, I guess, but i believe that most plant observations, even those from people with hundreds or thousands of observations, have just a single photo. It’d be interesting to see what the actual statistics are. New observers often post a single photo, because they’re using Seek to post, or just don’t realize that more information is needed for a good ID. Many long-time experts just post a single photo because they know what they saw, and might not care so much whether there’s really enough info there for another person to be able to make an ID.
2 Likes
Making a bad observation, like this, is its own problem, and not really relevant to this discussion. If anything, the muddled ID you’d get by including all of the photos in the ID for such a case ought to help the observer see that they need to separate their photos into multiple observations.
It would be interesting, but because all observations can’t be with multiple photos, realistically, we have to decide how many is enough, it’d be a hard process if we could do it. But if there’s at least one such observation, probably it means user already would like the system to check all the photos in that one observation, I’m sure it’s also possible to create it that way so it can make a decision if one photo gives completely different results from the others it won’t be counted or suggestions for it will be shown separately.
1 Like
My impression is that new iNatters offer single images. But many people who are interested in plants, if they want an ID from iNat, learn to bring adequate / multiple pictures. At least flower / fruit, leaf, wide view. Yesterday was a Pelargonium - not going to species unless you bring us leaves for that flower - said the experts.
3 Likes