Sexual dimorphism and computer vision: problems and suggestions

Hi,

I like to point out there is a bias in the computer vision tool currently used for species that show sexual dimorphism. Or maybe in the way the data is put in.

In a lot of Syrphidae (Flower or Hoverflies) show sexual dimorphism and need to be identified separately (:male_sign: :female_sign:) by the computer vision tool (is this already the case?)…

As example I am using Syrphus rectus.. A common species in the US. By looking closely to the recorded species mainly females (±95%) are ID-ed. (eyes separated)
Where are all the males (eyes connected)?
(Ps. note that in some Syrphid tribes male eyes are separated too! )
Most of the males can be found one level up in the tree under Syrphus but going through all thousand photos is almost impossible.

In the Syrphus rectus example: If you select only males you can find for instance some females between the photos. That needs to be corrected properly (by curators/identifiers/computer vision).

What to do?

Some suggestion I can come up with:

Curators/Identifiers should play more attention while identifying species that the gender is set correctly for species where this is important.

Computer vision can separate them properly.

Technically make it easier for curators/identifiers to set the gender :male_sign: :female_sign: directly in the overview as mostly it is possible to see it there already (even Agree-ing in the overview is possible most of the time).

Furthermore it would be helpful for identifiers/curators to filter out species identified only to genus/tribe/subfamily level. Now all species under a tree branch are shown (Nobody? will be going through them one by one to see if they are already identified to species level, especially with the common ones).

Cheers,
Bastiaan

This isn’t a bug so I moved it to General.

I don’t really see this as a problem in the specific case, since it is not really possible to distinguish male S rectus from S vitripennis morphologically, even with a specimen. It is a good thing if the CV only suggests females for the species. It does look like a few observations need corrections though.

Hmm, Maybe this wasn’t indeed a good example.

Some of the things you are describing can already be done with search filters on the Identify page!

For example, here are all the observations from genus Syrphus (just at the genus level):
https://www.inaturalist.org/observations/identify?taxon_id=52489&lrank=genus
(If you click on “filters” you can see how I did that.) You can also use search filters to look for observations that have not been annotated yet.

Also check out this tool (it’s a browser extension):
https://forum.inaturalist.org/t/announcing-the-universal-metadata-tool-beta/53182
It’s unofficial, but it would allow you to easily create a button to click to automatically identify the species and annotate it “male”.

I had a look and the vast majority were females (a couple mis-sexed by myself, oops!).

The general point is interesting, The CV does seem to cope with dimorphism, and even, increasingly with larvae. I don’t know how. In general it would be good if more observations had sex annotations. It would be much easier if the keyboard shortcuts worked on the identify page without having to change tab first, and also if they worked in observation view. At the moment the proportion with annotations in most parts of the world is too low for the CV to do anything with, but if it were otherwise, perhaps taking annotations into account could improve the accuracy.

1 Like

Super, thanks for the tips @taylorse this is really usefull indeed. I will try to see if the addon will work on my Firefox

In my experience, alas, it does work on Firefox, but you have to use a workaround to get it to load as a temporary extension, and you have to do that again whenever you close your browser. (There is a process to make it possible to install it permanently, but that process does not currently work.)

It’s much easier to use on Chrome. I now identify on Chrome for that reason.