Chrome extension showing Computer Vision confidence

i’m sort of surprised that there’s only one review for the extension. the review does provide some interesting feedback about the app not working for partner sites like iNaturalist.ca. seems like that would be an easy tweak though?

I just noticed that, too! I’m not very good at paying attention to what’s going on with the extension, apparently :upside_down_face:

It should just be configuration to make it work on other TLDs, I think. I’m planning on checking it out this weekend.

3 Likes

FWIW, the color-coding does make that lack of confidence pretty clear:

You probably already know this, but cropping the image, even if you don’t upload the cropped version (though I usually do, especially if it’s unclear what the focus is), usually really helps:

2 Likes

any idea why you got the ground cricket suggestion from the uncropped version when I didn’t? it looks like the other suggestions were different too; scorpion wasn’t on th elist for me. It looks like you kept the geotags so I’m pretty confused why we’re getting different results from the AI

if you download the 2048x1365 px (“original”) version of your photo from the system (https://inaturalist-open-data.s3.amazonaws.com/photos/226047732/original.jpeg) and try to upload that, you should get suggestions similar to what sessilefielder got (since i assume is what was used by sessilefielder to test).

your actual source image is a much bigger file, and you may get slightly different suggestions because it’s a slightly different image than that smaller image. this difference is especially noticeable since the computer vision has very low visual match scores for all its suggestions for both photos.

2 Likes

'cos we are too busy using it!

1 Like

can’t test it, since now it has your species it is choosing among scorpions.
But I would have started with a broad ID of scorpions, then ask CV to prompt me further.

Thank you very much for getting it back up!

What you mention about the scoring is what I expected it to be. What I was trying to get at was that I would much rather there wasn’t a step change between different values (i.e. I only see a red or green distinction currently) and would much prefer to see the values themselves so I can decide myself at what point the distinction should apply.

Also, if I check the box to “Enable color-blind mode” it doesn’t do anything at all. I.e. the red-green colours disappear, but no alternative colours appear to replace them, so it’s the same result as having the extension disabled. Since there are a range of different kinds of colour vision variations among people, I think it would be better to simply label different options with the different colour scales they respectively use. This is another reason I’d much rather have an option to just display the underlying value itself. Colour scales are generally problematic for accessibility.

P.S. What does it mean if I get a suggestion appearing without any colour on it at all as follows:
https://forum.inaturalist.org/t/new-computer-vision-model-released-august-2022/34588/13 ?

2 Likes

I would much rather there wasn’t a step change between different values (i.e. I only see a red or green distinction currently)

It’s a continuous spectrum of hues for HSL; you may just be looking at observations where there is a clear “winner”, so you get a really solid green for the first suggestion and really solid red for the rest. Here’s an example where it’s less certain:

I check the box to “Enable color-blind mode” it doesn’t do anything at all.

That’s because I’m dumb and I broke it. I’ll publish a fix in the next version, but this is what it looks like:

What does it mean if I get a suggestion appearing without any colour on it at all as follows

Probably that the score from the API is invalid in some way and I’m not handling that case correctly (or maybe this is the desired behavior, dunno–have to think about it). I’ll look into it and reply in that thread.

3 Likes

Ah, actually I see what’s going on, and it’s on my end.

How the extension works on the upload page is less than ideal. It gets the scores by hooking into the live scoring response, but the app caches those requests, so that hook only gets triggered the first time you score an uploaded image (by clicking on the “Species name” input field, which triggers that scoring request). If you score another image and then go back to an earlier one, the extension loses the earlier one’s scores and is actually coloring (or not) based on the scores from the subsequent image. When a suggestion is present for that first image but missing from the next, it will have a null score in that scenario and thus no color. Of course, even if it had a color, it would be being applied to the wrong image, as the score was generated for a different image.

I remember being displeased with this behavior when I originally wrote the extension a couple years ago and there being some roadblock to getting around it, but I don’t remember the specifics. When I have time I’ll dive into it again and see if there’s a good solution.

3 Likes

This discussion was becoming its own beast so I extricated it from the original topic. I think I got everything…

1 Like

Is this even a possibility in the near future @tiwane?

The extension exists. You can use it. I have relied on it since it was first built.

This was a fun one. I think it should work better now, remembering each photo’s CV results (as long as the filenames are distinct). It’s live in version 0.4.0; let me know if you notice any issues.

1 Like

Thank you! I have noticed the update now with the 0.4.1 release (since it required confirmation of permissions for the new websites added). The “color blind” mode does now appear to work giving a green (high) to blue (low) scaling. This works well enough for me. If you are ever considering adding more in the future I would suggest utilising the entire colour spectrum going from blue (high) to red (low) for greater clarity to cover the broadest range of users (or simply displaying the numerical values for maximum accessibility). Though no worries at all if there is not enough time or demand for this. It is already brilliant as it is!

2 Likes

Finally got around to this. Support for all sites in the iNat network is now live in 0.4.1.

4 Likes

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.