Computer Vision should tell us how sure it is of its suggestions

yes. i think that’s exactly why showing the underlying scores would be enlightening – because the #1 choice in a list of bad choices is not the same as the #1 choice in a list of good choices. so if you can see the actual scores, you have better insight into whether you’re being presented good choices or if you’re being presented bad choices.

just for example, here are 3 of my own ant observations, along with the actual computer vision scores:

  • https://www.inaturalist.org/observations/43079992 (needs ID as Myrmicine Ants, possibly Bicolored Pennant Ant):

    we’re pretty sure it’s in this family:

    • (99.9) Ants (Formicidae)

    our top suggestions (combined score, vision score):

    • (40.2, 53.4) Dolicohoderus genus
    • (10.2, 4.5) Bicolored Pennant Ant (Tetramorium bicarinatum)
    • (5.6, 7.4) Acorn Ants (Temnothorax genus)
    • (4.6, 6.1) African Big-headed Ant (Pheidole megacephala)
    • (4.4, 1.9) Graceful Twig Ant (Pseudomymex gracilis)
    • (4.2, 1.8) Florida Carpenter Ant (Camponotus floridanus)
    • (3.2, 4.3) Arboreal Bicolored Slender Ant (Tetraponera rufonigra)
    • (3.2, 1.4) Crematogaster laeviuscula
  • https://www.inaturalist.org/observations/20826798 (research grade Red Imported Fire Ant / Solenopsis invicta)

    we’re pretty sure it’s in this family:

    • (100) Ants (Formicidae)

    our top suggestions (combined score, vision score):

    • (50.0, 30.5) Forelius genus
    • (8.3, 15.4) Asian Weaver Ant (Oecophylla smaragdina)
    • (6.0, 11.1) Mediterranean Acrobat Ant (Crematogaster scutellaris)
    • (5.7, 3.5) Argentine Ant (Linepithema humile)
    • (5.5, 10.1) Azteca genus
    • (3.5, 2.1) Crematogaster laeviuscula
    • (2.6, 4.9) Tropical Fire Ant (Solenopsis geminata)
    • (2.6, 4.8) Yellow Crazy Ant (Anopolepis gracilipes)
  • https://www.inaturalist.org/observations/44126816 (research grade as Eastern Black Carpenter Ant / Componotus pennsylvanicus):

    we’re pretty sure it’s in this genus:

    • (77.9) Carpenter Ants (Camponotus)

    our top suggestions (combined score, vision score):

    • (64.3, 61.3) Shimmering Golden Sugar Ant (C. sericeiventris)
    • (7.0, 1.8) Eastern Black Carpenter Ant (C. pennsylvanicus)
    • (6.9, 7.3) Giant Turtle Ant (Cephalotes atratus)
    • (4.0, 4.3) Eciton genus
    • (2.6, 2.8) Bullet Ant (Paraponera clavata)
    • (2.0, 2.2) Diacamma genus
    • (1.9, 2.0) Giant Forest Ant (Dinomyrmex gigas)
    • (1.8,1.9) Hairy Panther Ant (Neoponera villosa)

if using sessilefielder’s red-to-green gradient, remember that:

so most of the “top suggestions” above would be red to yellow, whereas the “we’re pretty sure” suggestion would be more green. hopefully in such cases, that would push most folks to select the green rather than the yellow or orange, if they were simply choosing blindly based on the system’s suggestions.

i also think if people could see that, say, bird suggestions tend to be very green, while, say, spider suggestions tend to be very red, then they would also be much more careful about relying on the computer vision for spiders.

or if they see two equally green birds suggestions, they might pause for a moment to consider why both are equally green before just blindly selecting the first choice.

of course, computer vision suggestions will never be perfect. there will always be mistakes, but i think showing the computer vision scores will help reduce (rather than increase) the likelihood that the community will adopt those mistakes.

9 Likes