In response to the ongoing discussion about the relative accuracy of the CV, I decided to pull together a truly randomized/representative set of observations and review the performance/use etc of the computer vision. Last Updated - 2021-08-29 (n=150 records) Notes on methodology records are rand…

[image] cmcheatle: review is restricted to research grade records. I understand and know that the performance is likely lower on needs ID records, but I’m not an expert on every taxa in the world, and I can’t evaluate if the users ID an/or a CV result for that record is correct Great to see e…

This and a related discussion inspired me to start my only little experiment with CV, for my own observations. Unfortunately, the photos I had readily available to give it are mostly heat-damaged conifers. Even from my small initial sample, I can report conclusively that the CV is really terrible …

A very quick look at discrepancy between Needs ID vs RG data. Just checking over the CV IDs on the most recent few hundred observations of UK Diptera, I found the following : NEEDS ID Correct = 41% Incorrect = 22% Unsure = 36% RESEARCH GRADE Correct = 93% Incorrect = 3% Unsure = 3% A big…

I found something similar with Tall Oat Grass, Arrhenatherum elatius. The “Needs ID” were identified as at least 10 different species, and many didn’t look all that much like Arrhenatherum. With some trepidation, I checked the “Research Grade” observations. All or nearly all were correct!

This sort of discrepancy seems common across a variety of categories. CV is accurate with sightings that are Research Grade, or are in North America and Europe, or are of vertebrates, but it is generally less correct with sightings that are Needs ID, or are in the southern hemisphere, or are of inv…

[image] cmcheatle: Once I use this do I then have to manually add html break points as it seems to have resultsed in a mismatch in the WYSIWYG editor whitespace between sections and what is rendered. Sorry, I don’t quite follow what you mean. This thread, which I probably should have linked t…

[image] sbushes: You´re also not checking if the RG records themselves are correct(?) A sizeable portion of RG records are likely incorrect in complex taxa If you have evidence that a meaningful percentage of rg records when chosen randomly from the global dataset (ie not intentionally picki…

I specifically said in complex taxa for that statement. Within more complex taxa, Lucilia sericata would be a good example I imagine. L.Sericata is the third most-observed species of Diptera globally in current rankings. I´m not sure what % would be “meaningful”… but for example, I can see at lea…

If you are going to review the performance of the CV versus human judgement, you kind of need to pick records where both are available. If people feel adding records where the CV was used by the observer and then a human disagreed with it leaving it at needs id will help, them I will add those when …

I think something’s wrong with your last table as the column headers are identical but clearly don’t refer to the same data. Or maybe I just can’t figure out how to read it.

Computer vision performance summary data

General

lawnranger August 30, 2021, 4:09am 29

Sorry, I don’t quite follow what you mean. This thread, which I probably should have linked to initially, may be helpful: Help needed to make the new Markdown code for Table work

Topic		Replies	Views
Problems with Computer Vision and new/inexperienced users General	132	5965	October 28, 2021
Comparing CV Outcomes in a Pair of near-Cryptic Moth Species General question	60	1399	October 24, 2024
Is it possible to use the computer vision to review existing research grade observations and flag those that may be incorrect? General	16	849	November 26, 2024
Recommendations on improving the AI algorithm? General	47	1055	April 13, 2025
Opinion on the computer vision icon? General question	60	2429	August 15, 2023

Computer vision performance summary data

Related topics