In response to the ongoing discussion about the relative accuracy of the CV, I decided to pull together a truly randomized/representative set of observations and review the performance/use etc of the computer vision. Last Updated - 2021-08-29 (n=150 records) Notes on methodology records are rand…

[image] cmcheatle: The other things I thought afterwards about adding is how many observations has the observer submitted to see if there is a difference in use between new or experienced users. I might suggest the sum of observations + identifications as a rough proxy for experience, since t…

[image] cmcheatle: review is restricted to research grade records. I understand and know that the performance is likely lower on needs ID records, but I’m not an expert on every taxa in the world, and I can’t evaluate if the users ID an/or a CV result for that record is correct Great to see e…

This and a related discussion inspired me to start my only little experiment with CV, for my own observations. Unfortunately, the photos I had readily available to give it are mostly heat-damaged conifers. Even from my small initial sample, I can report conclusively that the CV is really terrible …

A very quick look at discrepancy between Needs ID vs RG data. Just checking over the CV IDs on the most recent few hundred observations of UK Diptera, I found the following : NEEDS ID Correct = 41% Incorrect = 22% Unsure = 36% RESEARCH GRADE Correct = 93% Incorrect = 3% Unsure = 3% A big…

I found something similar with Tall Oat Grass, Arrhenatherum elatius. The “Needs ID” were identified as at least 10 different species, and many didn’t look all that much like Arrhenatherum. With some trepidation, I checked the “Research Grade” observations. All or nearly all were correct!

This sort of discrepancy seems common across a variety of categories. CV is accurate with sightings that are Research Grade, or are in North America and Europe, or are of vertebrates, but it is generally less correct with sightings that are Needs ID, or are in the southern hemisphere, or are of inv…

[image] cmcheatle: Once I use this do I then have to manually add html break points as it seems to have resultsed in a mismatch in the WYSIWYG editor whitespace between sections and what is rendered. Sorry, I don’t quite follow what you mean. This thread, which I probably should have linked t…

[image] sbushes: You´re also not checking if the RG records themselves are correct(?) A sizeable portion of RG records are likely incorrect in complex taxa If you have evidence that a meaningful percentage of rg records when chosen randomly from the global dataset (ie not intentionally picki…

I specifically said in complex taxa for that statement. Within more complex taxa, Lucilia sericata would be a good example I imagine. L.Sericata is the third most-observed species of Diptera globally in current rankings. I´m not sure what % would be “meaningful”… but for example, I can see at lea…

If you are going to review the performance of the CV versus human judgement, you kind of need to pick records where both are available. If people feel adding records where the CV was used by the observer and then a human disagreed with it leaving it at needs id will help, them I will add those when …

Computer vision performance summary data

General

jdmore August 29, 2021, 8:36pm 22

I might suggest the sum of observations + identifications as a rough proxy for experience, since there are some experts who submit relatively few observations of their own.

3 Likes

Topic		Replies	Views
Problems with Computer Vision and new/inexperienced users General	134	5535	December 27, 2021
Comparing CV Outcomes in a Pair of near-Cryptic Moth Species General question	61	1257	December 23, 2024
Is it possible to use the computer vision to review existing research grade observations and flag those that may be incorrect? General	17	800	January 25, 2025
Recommendations on improving the AI algorithm? General	50	937	June 12, 2025
Opinion on the computer vision icon? General question	61	2327	October 14, 2023

Computer vision performance summary data

Related topics