First of all, I think we should decouple the feature request for automated annotation from the ongoing vision system. To us, it doesn’t make sense to talk about them together, because we won’t be implementing them together. Happy to talk about that further if you’d like.
With regard to thresholds, the unfortunate answer is that we don’t know what the thresholds are yet. We just know that more data produces better models. It’s taken us a few years to be able to understand and chart the relationship between the number of ID’d photos of species and suggestion accuracy. We haven’t even started looking at how an annotation model would perform, or what reasonable thresholds for one might be. It may be that the thresholds are similar to suggestions, but we don’t know that yet.
I know this is an old topic, but would this mean any possible use of CV for, for example, flowering phenology annotations would be based on all observations of plants rather than on a taxa-by-taxa basis? Sounds intriguing if so, though I could see problems with galls or irregular growths.
It’d probably need a tiered implementation as well, with a different level of confidence applied if it has been solely confirmed by the CV.
I was prompted by @annkatrinrose to work thru my own obs, and now remember to annotate Flowering as I upload. That would be an easy task for newbie volunteers.
But we absolutely need to be able to annotate individual photos. If I ask for pictures of fruiting wotsits, I get. All the pictures. From any obs. Which includes a fruiting picture among oh so many to slog thru! Not user friendly. At least the original request for ‘caterpillar’ works as pupa and adult must, time wise, be separate obs.
Another PS since I read the older comments today. I am obsessed with IDing Unknowns. My moth / butterfly knowledge is small, but I can see immediate taxon specialist response to - it’s a Lepidoptera. While I am there, I annotate as larva where it fits (which is also appreciated by @karoopixie)
This is my first post in the Forum, so I hope this discussion belongs here.
I am using iNaturalist almost daily, and try to add Annotations to the best of my ability to each organism I observe and identify. As a researcher, I think that recording many observations of many organisms is a good thing, but that they must be as complete as possible, so that they can one day be used by someone for a project of some kind. But it takes time to add 3 or 4 annotations to each observation, even though the “batch edit” option helps a lot.
So, I was wondering if the iNat AI could be trained to add by itself some annotations (sex, evidence of presence…) when it is possible. It is obviously impossible for a lot of organisms, but as an example I am IDing a lot of spiders, insects and sometimes birds, which may (not for all !) easily by sexed. I know that AI is not perfect, but adding a kind of “research grade” to annotations that just have to be confirmed by 1 click from AI - or not - would greatly improve the quality of observations from my point of view.
Do you know if some kind of AI training as already been tested on iNat ?
Also, do you think it would be useful ? If not, why ?
It’s an interesting idea, at least for those cases that AI could be expected to manage almost flawlessly (eg: stage of life for Lepidoptera). Any such thing that could be automated surely would contribute by freeing up identifiers’ time to tend to matters that AI is less capable of handling.
It’s a bit of a vicious circle – to be able to recognize and annotate life stages correctly, the AI needs a suitable quantity of annotated material. So it seems to me that there is no way to avoid the necessity of humans annotating observations to start with.
Another issue is that it is currently not possible to correct annotations unless you are the observer or annotator. It seems to me that this would need to be fixed before any automatic AI annotation could be implemented. (I think automatic annotations would also need to have a different status than human-added ones, to make it possible to review them.)
(Given how much the CV struggles with hymenopterans, I wouldn’t trust the AI to annotate bees correctly, in spite of the fact that they often have distinct sexual dimorphism. I do, however, wonder whether incorporating annotations into the CV training might potentially result in improved suggestions – but again, this would require a suitably large annotated training set.)
Technically, we currently have a trial for that concept within iNat itself. Many, Many, users just use the suggested ID, and then others have to come along and correct it. Frequently the original identifier (usually the observer) doesn’t notice or doesn’t withdraw meaning that more people are needed to correct it. Add to that that the annotation page is on a different page from the identify page and you’re potentially adding a lot more work. I usually don’t add annotation since they are rarely used by observers and so less in need of being corrected (not always to be sure). I feel that this would not free up time, but take up time, by encouraging people place an “ID” where before they straight up “didn’t know”. Now I’d have to double check another level on every observation.
So the only way I’d support this, is if annotations had a similar RG system that faulty ID’s can be corrected.
Oh, and must send a notification to people on the observations! I have gone through clearing out hundreds of bad annotations, and I found out that no one gets a notification for those, so I then had to add a comment to all of those observations.
Theoretically a great idea. However, we’d need to change how we correct annotations (as noted above) and getting a good base of specimens for training would be a challenge. Might have to be implemented taxon by taxon. I’m ambivalent about actually implementing this. (Fortunately, I won’t be involved with that, if it happens.)
This would be the only way I’d vote for this- I’d want to see it rolled out on a small group of taxa first.
My main concern is that which annotations are relevant/correct depends upon the identification of the observation. So if the CV is suggesting 4 possible species for an observation, here are just a few conundrums that could happen:
-the CV recognizes that the plant is brown and crispy, which could mean “dead” if identified as an annual or “alive”/“can’t tell” if identified as a perennial.
-the CV recognizes a wormlike animal which could be “adult” if identified as an actual worm or “larva” if annotated as an insect
-the CV recognizes a leaf mine on a leaf, which would be “larva”/“leafmine” if identified as an insect, but not those things if identified as the plant
Those are just a few examples, but my point is I think it will be challenging to work out a way for the CV to suggest these annotations, when it has to know what taxon you’re observing first in order to get the annotations right. We obviously don’t want a situation where the CV recommends “larva” before it knows if you’re observing something that even has a larval stage. Either you’d have to have the ID suggestion box include the annotation in what you click on when you take the CV identification (ie top suggestions says “Monarch larva” to click on, rather than just “Monarch”), or you’d have to have a separate field to click on “annotation suggestions” after entering the identification.
In any case, the “IF species A, THEN annotation X, but IF species B, THEN annotation Y” question has to be addressed. If the CV is given that the photo shows a monarch, then annotating it as adult or larva seems like really easy photo recognition. If the CV isn’t given what the identity is as a starting point, then CV annotations might be a huge mess.