Fortunately for me, it works almost perfectly on many plant taxa in California. Why? We get out what we put in: species which are both commonly observed and commonly identified by humans are the ones the CV knows the best. The California plants CV doesn’t work on are either rare or ones we humans can’t ID from photos. (I know Barbara is aware of that–so sorry about the grasses!) Of course we’re spoiled in California because iNat is based here; the pool of identified photographs ready to train the CV was large before the CV even came into existence. I have not been around long enough to remember the time before CV, but I bet California was able to skip over a lot of that awkward phase during which CV knows just enough to be dangerous.
8 Likes
Absolutely the decision to implement the computer vision model was a pivot point in the growth trajectory of iNaturalist. As I said above, I have not been around long enough to remember a time before CV, but I know without doubt that iNat was a different place without it. These days the majority of users who download the app do so to try out the CV. (Or because their school teacher made them do it, but that’s another discussion!) They want to point their camera and instantly get back a name.
5 Likes
I would suggest that allowing people to present the Computer Vision suggestion as their own provides no benefit to iNaturalist. I think that a good solution would be to always automatically provide a Computer Vision suggestion for any observation that doesn’t have a Research Grade ID, but to provide it explicitly as a Computer Vision suggestion, not associated with any user.
The computer algorithm generates a suggestion regardless of whether or not someone clicks on it. A user selecting the Computer Vision suggestion is providing no additional information. It is only providing misinformation, as it is implying that some human that was familiar with that species looked at the photo and decided that they could identify it to that species. Which is not true, because if they required the Computer Vision suggestion then they could not ID it, and therefore should not be suggesting an ID.
Yes, I know that some knowledgeable users might make use of Computer Vision to speed up their IDs because they don’t have to type as much. However, the autofill function already does that. And I would argue that such an user isn’t really contributing to a good ID, because there is no way to tell if they really knew their ID or were just agreeing with the Computer Vision suggestion.
Also, allowing people to select the Computer Vision suggestion allows multiple people to do this, which is just allowing the computer algorithm to have multiple votes. And it means that these Computer Vision suggestions are then used to reinforce the computer algorithm, which is a circular logic that reinforces errors.
Following this logic, iNaturalist could remove the Computer Vision suggestion entirely from the interface that people are shown when they suggest an ID. It can still autofill, which speeds up your ID, but no Computer Vision suggestion.
iNaturalist could automatically always provide a Computer Vision suggestion, in a separate section that was underneath the photo but above the “Activity” section. This would be clearly identified as a suggestion from the computer algorithm. It would not count towards Research Grade status, and it would not be used to reinforce the computer algorithm.
People who were just using iNaturalist to instantly get a suggested ID for a photo that they uploaded would still get this, and they wouldn’t have to suggest an ID for a species that they didn’t know. They wouldn’t actually have to suggest any ID if they didn’t want to, the Computer Vision suggestion could be used to temporarily give the observation a name (in the same way as a “Needs ID” observation is still given the name of the best guess).
The Computer Vision suggestion could be removed when a Research Grade ID was acheived, or it could just be left there. I think that it would be quite interesting to always be able to see how it was performing. I think that it would help to show people how much they should be trusting the computer algorithm, as all of its successes and failures would be clearly on display. It is an amazing tool, that performs way better than I would think possible in many situations. But there are also a lot of situations where it completely fails.
6 Likes
If I remember correctly, this was for test images of the set of taxa included in the CV model. It didn’t include images of the vast majority of taxa that are not in the model.
7 Likes
This discounts the unknown but sizeable group of people who use it to speed up entry knowing full well what the species is. It also assumes every person who agrees with a CV suggestion is blindly doing so.
Just because someone chose to use the CV to identify that Northern Mockingbird (whether they were legitimately unaware of what it was or not) doesn’t mean that if I agree to it because it is correct I am giving the CV another vote.
7 Likes
Yes, I like it. It is very similar to an idea I had about how to eliminate unknowns.
4 Likes
I think you are onto something there. I use iNat for my personal observation reporting in the absence of much else (I use eBird for birds) however, because I happen to be a real biologist with alphabet spaghetti I get a lot of requests form people in my local community to help with ID-ing something they have seen. Just being a biologist doesn’t make me an expert on IDs across the board whatever the pub lic might think, so for creatures I am only dimly aware of I often use iNat’s AI system as a jumping off point and that usually works because I know where to look for the additional detailed taxonomic stuff. What is irritating is when, after some effort, you get back to people and say “I think it is species X” or “your photo really doesn’t help me get closer than family or genus” and they respond with “thanks - so, it’s a spider then. Cool”.
5 Likes
This is not, or should not be, a “game”. Data matters.
2 Likes
Yes, some knowledgeable users make use of Computer Vision to speed up their IDs because they don’t have to type as much. However, the autofill function already does that, and it is pretty quick.
The problem is that it is impossible to tell if this user really knew the ID of the observation, or they were just agreeing with the Computer Vision suggestion. That means that all Computer Vision suggested IDs are ambiguous, we don’t know if they were contributed by someone who was actually able to identify the species, or if it is just another Computer Vision suggestion. If the user did actually know what the species was, the iNaturalist interface is effectively sabotaging their ID so that we don’t know if it was actually a human ID.
1 Like
For some common plant species in California, the CV is close to or exceeding what actual experts could do from a single picture; keep in mind the CV is only allowed to look at the first photo in an observation! I’ve seen hard IDs where all the traditional key characteristics are only in the second or third photo, and an expert could hardly ID to family from just the first, yet the computer vision is ‘pretty sure’ on genus and its top suggestion is correct (despite several common close lookalikes), as verifiable by the later pictures in the observation.
To my mind, the solution to the computer vision problems are just to keep doing what we’re doing; fixing the errors and pushing more and more rare species over the 100 observation threshold. Computer vision makes this faster not just because of autofill but because it makes it practical for a single person to clear unknowns to family or genus with >90% accuracy for most of the tree of life. I’d hazard to guess perhaps a few people in the world could do that as well or better, and the time of people like that is much too valuable for them to spend all their time clearing unknowns, because presumably they could also be clearing hard taxa to species.
Most if not all of the taxa were the problem with CV is not insufficient number of observations or insufficient experts to classify them are ones where humans would struggle too, so eliminating CV does nothing to improve that and probably makes it worse.
The biggest improvement to the interface to mitigate mis IDs to species is probably just to make it more aggressive about offering suggestions at higher taxonomic levels, which I think is already in progress.
4 Likes
I dont see the benefits of making it harder or slower for knowledgeable users to enter their information. Power users are likely contributing a majority of the recrds to the site, removing functionality from them to offset new or unengaged users behaviour seems like a step backwards. Entering observations is already time consuming enough, especially on the mobile apps.
If the person who knows what it is uses the computer vision and they/it correctly identify it, what is the issue of if they used a faster method (CV) or slower method (manual entry) of the taxon name?
For example, the province I live in has about 2.9 million records submitted. They come from 59k distinct observers. But the top 100 observers in terms of number of records submitted account for a whopping 35% of all records submitted in the province.
Slowing their entry down is not helpful, the alternative of saying they can continue to use the CV if they wish, but then that will not count towards the community ID just adds a large burden onto an already overburdened identifier pool.
7 Likes
You do not have to manually enter the species name, the autofill already makes it a lot faster. Yes, not quite as fast as the CV, but still very fast.
Making it slightly faster is not valuable if the end result isn’t useful. Adding an ID that says “This is what the computer algorithm says the species is” is providing no additional information to iNaturalist. For a very significant portion of the CV suggested IDs provided on iNaturalist, that is all that the user is saying, which is not only useless information, it is actually detrimental to the value of iNaturalist IDs and the effectiveness of the Computer Vision itself.
Yes, when some users are clicking on the CV suggested ID, they are saying “This is what the computer algorithm says the species is AND I could see the features in this photo to reliably identify it myself AND I agree with the CV ID”. However, there is no way to differentiate between these different users, without creating a ranking system where the IDs of some users are systematically discounted.
I am completely in favour of having the computer vision suggest IDs, and I think that this adds a very powerful tool to iNaturalist. However, this is not a person, and it should not be conflated with a person’s ID. Currently, there is no way to tell if the ID was provided by the AI, or by a person.
2 Likes
I mentioned this higher up, but there is a way to tell: see point 4 in the computer vision section of the Q&A to see what it looks like: https://www.inaturalist.org/pages/help
There is no way to tell if the user was using it as an autofill or actually knew, because how could there be?
1 Like
I’m not certain that I really understand your argument, but if I am understanding correctly, I think that my suggestion would actually speed up the entry of observations, not slow it down. I am suggesting that the Computer Vision suggestion be automatically included in all observations that are uploaded. The user wouldn’t have to click on a single thing. I am actually saving them one click of the mouse or tap of the finger. It is valuable information, and should always be included, we don’t need to waste someone’s time to ask for it.
However, I am suggesting that this CV suggestion should not be provided as someone’s ID. It should be provided as a separate thing, that is clearly identified as a suggestion from the computer algorithm. It would be between the photo of the observation, and the “Activity” section.
4 Likes
Exactly. This is why it is not useful information. There are various ways that this could be differentiated, but not within the current interface.
One option is that the interface could just ask the user when they selected the computer vision suggestion if they actually could ID the specimen, or if they were just agreeing with the CV suggestion. However, I don’t think that this is the best option.
I think the best option is to not conflate computer algorithm suggestions with human IDs at all, always automatically provide the Computer Vision suggestion for each observation, but as a Computer Vision suggestion, not as a person’s ID.
1 Like
That’s fine for well documented species, and places in the world with good coverage. It’s going to create a nightmare of issues in taxa not well documented, not easy to evaluate with photos or places with lower rates of coverage on the site.
That’s going to just add to and likely create more problems as users insist on ‘well the computer says it is genus Setophaga’, so I’m going to ignore feedback from humans who say it is Setophaga magnolia. To say nothing of the CV is insisting on identifying the flower, when I want to document the bee on it etc.
If every record shows on it what the CV thinks it is, the level of and complaints about people blindly following the CV will skyrocket.
It seems like this is trying to solve a problem with relatively small (but highly visible) impact. To invent iNat’s version of Drake’s equation it seeks to resolve records impacted by
- what percentage of records are identified by users using the CV
- then eliminate those from above where the user actually does know what it is and is trying to be more efficient in their entry
- then find the ones where a faulty ID has been accepted and agreed to by someone else in order to reach research grade (if the CV was not there, the observer was either going to make a guess anyways, or leave it at some high taxonomic level) which both place an equal burden onto identifiers.
4 Likes
Pollinators, galls, parasites, bird nests in trees, the case I saw a couple days ago of some Chives growing through Angelica leaves where the user didn’t know what either was or even that they weren’t the same plant. In that case because both are IDable, so if the CV had forcibly autosuggested anything more specific than angiosperms it could have become an RG obs of that and the user might never have known what was going on… Yet in most of those cases it is relatively easy for a human to tell what is centered or what the more interesting obs would be. For those cases the list of CV suggestions is very helpful but whatever the top one happens to be does not.
1 Like
Take in mind please not everyone can even type all the time, I upload mostly with 1 arm and don’t need to type in most cases, I get problems and pains from typing, so autofill is not the solution.
4 Likes
Asking people if they can ‘explain why it is this or did you just accept what the computer said’ might not help your issue, if they can click yes, next.
Now - when a person accepts the computer suggestion, you have a human layer. Your version would be fully automated, without even the most basic
No - that rock - is NOT a seal.
3 Likes
Great discussion, thank you all. A few comments:
-
I suspect most people using CV are the observers, when they enter their own observations (via the app or web interface), because they are prompted to say something about what the species is. In contrast: power identifiers will be using the “Identify” tool on the web interface (which does not bring up CV). Most observers are not experts, which is a very good thing - it means iNat has mobilised a lot of people to look at nature and add records of it to a centralised database. The whole point of CV is to make the entry of records as easy as possible to observers, and that is how it should be. What we need to ensure is that errors added at the entry point are efficiently corrected! I am fine with observers making mistakes, what annoys is people who systematically make wrong identifications.
-
Using CV when entering an observation is not just about auto-fill, it can be of good help to remember the species’ name. I may perfectly recognise a species and yet not quite remember its name; or maybe I’ll remember the name but not exactly how to spell it (is it Turritellinella or Turrittelinella; Mimachlamys or Mymachlamis?). If I can see the right species among those proposed by CV, it is much faster to pick it than to go and check the name elsewhere so that I can type it.
-
CV often proposes multiple options, and picking among those is a human decision. The little symbol that is associated with an identification based on CV appears irrespective of whether the user picked the 1st or the 6th option on that list. Hence: don’t assume that when you see that symbol it means people blindly accepted the first thing CV proposed.
3 Likes