Any interest in a browser extension which translates the computer vision scores into a red-green scale? I slapped together a prototype today after reading this thread; here are some examples:
(Obviously) I am not a designer, but it does give a rough idea of what is possible with a purely client-side solution. If there’s interest, I’ll publish it and put the code on GitHub so others can help refine it.
What would be the thresholds for each color (is green 80% - 95%, or 70% to 95%)?
Does red sort of imply wrong so implicitly in our current visual lexicon that people don’t even consider the other suggestions (sometimes the right suggestion is further down the list)? I’d assumed that’s why explicit disagreement is an orange button instead of a red one, but I could be reading too much into that.
Would the colors be adjusted by the user for accessibility (r/g color blindness, etc)?
I’m not saying it’s a bad idea; these are just the first things that popped into my head when I looked at the pics (especially red = stop).
What would be the thresholds for each color (is green 80% - 95%, or 70% to 95%)?
It’s just the computer vision score, which is [0,100], scaled to [0,120] and used as the hue in HSL.
Does red sort of imply wrong so implicitly in our current visual lexicon that people don’t even consider the other suggestions (sometimes the right suggestion is further down the list)?
Red-green was just the first gradient that occurred to me. I don’t have an informed opinion about whether red in particular imparts a bias or, if so, whether that bias is relevant given that the extension would be opt-in.
Would the colors be adjusted by the user for accessibility (r/g color blindness, etc)?
This is a good point, and taken with your second one argues for a different color gradient, or at least a color-blindness mode.
In general, whether it’s color-coding or just exposing the raw number, as OP requested, I think it’s nice to be able to distinguish between different the different cases illustrated by my examples. Then again, I confess that in addition to not being a designer, I’m also not a statistician, and thus @kueda’s comment above is a bit lost on me:
I’m told the score should not be considered a metric of “confidence” or “probability” and it should mainly be used for ordering outputs
i.e. I’m unclear on whether the magnitude of the differences in scores is relevant in any way.
i think this is great as a proof of concept. i wonder what this would look like implemented as a color gradient that transitions to pure white by 70% of the width of the box (or maybe hsla that drops the alpha to 0 by 70% the width of the box)? or maybe more simply as a colored left border (or maybe left and bottom border)? maybe a gradient with variable points at which it transitioned to white (effectively creating something like a bar chart) could solve the color blindness problem?
then the main unresolved problem would be the problem of the situation where the real answer isn’t actually in the list. hopefully those kinds of situations would more often look more like your bottom example with lots of red than your top example above with bright green choices…
i think 0-120 hues on hsl is a good choice for color scales. on a scale of 0 to 1 then, 0 would be red, 0.25 would be orange, 0.5 would be yellow, 0.75 would be yellow-green, and 1 would be green. (i’m a little surprised to see so much red in the examples above, but maybe i shouldn’t be. this is exactly the kind of truth i was hoping could be revealed by something like this though.)
Great ideas. Here are a few examples based on them. Transition to white at 70% width:
then the main unresolved problem would be the problem of the situation where the real answer isn’t actually in the list. hopefully those kinds of situations would more often look more like your bottom example with lots of red than your top example above with bright green choices…
I think this is already a problem, given the “We’re pretty sure…” text (in fact, I don’t understand why that feature exists at all if what @kueda said above about the score only being good for ordinal ranking among suggestions is true). For example, computer vision is pretty sure that this golden paper wasp is in fact a Syngamia moth, and the UI is already heavily influencing people to choose that ID. What the colors would add is an addition push to select the species Syngamia florella:
then the left border bar option - I prefer to be able to read all the text on white, not against colour. The species names and Nearby need to be obvious.
Or with the colour gradient on the far right, where it is simply View all the way down.
I also like the left side bar…and then all that extra blank space between the end of the taxonomy name and the “view” could hold a probability as in @psium 's previous post.
The Chrome extension is published, and the code is on GitHub. You can choose between the two different display modes (sidebar vs. gradient), as well as a color-blind mode that changes the range from 0->120 to 240->120 on the hue spectrum.
I’ll work on a Firefox version as well.
Thank you - I will try it.
Tanks, that’s interesting. I played with some observation to actually measure how it performed. I’m surprised by how good the software is at avoiding false high confidence positives which were my biggest fear. But there are still some errors. Take this recent ID : https://www.inaturalist.org/observations/52616986 . The CV predicts a Formica at 99.2%, the picture is of decent quality without any disturbing pattern in the back or foreground and I think can allow to refine up to a species complex. It is not a Formica, but a Crematogastrini, which is very far off.
I still find color code to be loaded with meaning (“red = don’t”, “green”=“go for it”), but that’s clearly a matter of taste, and I totally hear you.
the color-blind option for sessilefielder’s browser extension uses a hue gradient that goes from blue (bad match) to green (good match). blue might have less of a negative connation than red.
it would also be technically possible to do a saturation gradient, where you could go from, say, green (good match) to gray (bad match). or maybe a saturation + lightness gradient, where you could go from, say, green (good match) through light greenish-gray to white (bad match). there are lots of different ways to represent the data with color.
if you have thoughts on the best way to represent the data, whether with plain numbers, or color, or something else, please feel free to describe your preferred approach.
I’m not an UX/UI designer nor a good data vizualiation artist, however I’d be happy to give feedback.
The white/grey to blue seems very culturally neutral at first glance with the advantage of being visible by everyone. The main disadvantage is see is that the function of the blue tip left bar is not obvious compared to a “green-red” or any “good-bad” multicolor scale.
This seems like a great idea, especially some of the color-coding suggested by @sessilefielder. I agree that perhaps a red/green scale wouldn’t be best, but I like the idea a lot. Is something like this still a possibility, or have staff decided they won’t move forward with it? (I suppose since the thread is still open that’s a good sign.)
I use that app on Chrome - with the red green colour scale
https://chrome.google.com/webstore/detail/inaturalist-enhancement-s/hdnjehcihcpjphgbkagjobenejgldnah
Cool! Is there a way to change the display though? It seems to automatically be the left-side bar, but I would prefer it to cover the entire suggestion.
EDIT: Never mind, I figured it out. I don’t have much experience with extensions.
Yes - you can toggle between my preferred sidebar
and your gradient across the screen.
(On the 3rd slide there)
Thank you for your work on this!!! It appears that the link to the extension is no longer working?
Also, would it be possible to simply display a numerical value (e.g. in place of the left hand colour bar, or anywhere convenient really), rather than a colour scale? Or even have both if others prefer that? I’m not a fan of having sharp distinctions drawn between different score levels since it can never have a consistent level of meaning across different observations.
Oops, looks like I missed an email from Chrome telling me they were yanking the extension because I don’t have a link to a privacy policy. I’ll try to get it back up ASAP and drop a note here when it’s back.
It would be possible to display the number, but note that the colors are not relative to a specific observation, i.e. anchored by the score of the top suggestion, but rather based on the absolute scale of the output scores.
