Use computer vision to annotate observations

Hi David,

First of all, I think we should decouple the feature request for automated annotation from the ongoing vision system. To us, it doesn’t make sense to talk about them together, because we won’t be implementing them together. Happy to talk about that further if you’d like.

With regard to thresholds, the unfortunate answer is that we don’t know what the thresholds are yet. We just know that more data produces better models. It’s taken us a few years to be able to understand and chart the relationship between the number of ID’d photos of species and suggestion accuracy. We haven’t even started looking at how an annotation model would perform, or what reasonable thresholds for one might be. It may be that the thresholds are similar to suggestions, but we don’t know that yet.

Thanks,
alex

11 Likes

I know this is an old topic, but would this mean any possible use of CV for, for example, flowering phenology annotations would be based on all observations of plants rather than on a taxa-by-taxa basis? Sounds intriguing if so, though I could see problems with galls or irregular growths.

It’d probably need a tiered implementation as well, with a different level of confidence applied if it has been solely confirmed by the CV.

2 Likes

This is a video about counting flowering flowers in 1m2 square. They just started it and i thought that other models already were annotatin caterpillars from butterflies.
(24) FLORON-dag 2022: Tellen & herkennen van bloemen met artificiële intelligentie - Gerard Schouten - YouTube

I was prompted by @annkatrinrose to work thru my own obs, and now remember to annotate Flowering as I upload. That would be an easy task for newbie volunteers.

But we absolutely need to be able to annotate individual photos. If I ask for pictures of fruiting wotsits, I get. All the pictures. From any obs. Which includes a fruiting picture among oh so many to slog thru! Not user friendly. At least the original request for ‘caterpillar’ works as pupa and adult must, time wise, be separate obs.

Another PS since I read the older comments today. I am obsessed with IDing Unknowns. My moth / butterfly knowledge is small, but I can see immediate taxon specialist response to - it’s a Lepidoptera. While I am there, I annotate as larva where it fits (which is also appreciated by @karoopixie)

3 Likes

But we do have
No Evidence of Flowering

which was added after the first CNC? And appreciated by rangers in our nature reserves, to help them ID plants out of flower / fruit.

PS your linked comment is from 2019 - so you can tick that as Done meanwhile.