But that is something they wanna achieve with the new image to text algorithm they are testing.
Calling it “image to text” isn’t an accurate description of what they are actually doing, and doesn’t give a realistic sense of what it is actually capable of. Really it’s more the opposite of that - it’s a text to image classifier.
It’s not dissecting images and describing their component parts, it’s classifying their similarity to the keywords in a human captioned training set - so when you search for “bird on a car”, it’s looking for the images with the best match to the ones trained with those caption key words.
And it’s still involving humans in the ongoing training to tell it when it returned something like what you asked for, and when it got that hopelessly wrong.
I don’t want to rain on your enthusiasm, but being realistic about how this works and what it is actually capable of is the best way to find genuinely good uses for it.
So maschine learning could quite well work on figuring out what a wing and what a thorax is.
That is one thing it could be trained to do, but if we’ve learned anything about “machine learning” and (more accurately) search and optimisation since circa the 50’s when some people asserted that “an electronic brain which could surpass Man” was just a decade and a few million dollars away - it’s that approaches which just try to mimic the way a human would do this are unlikely to work as well as those which better exploit things that the machine is actually good at doing …
So that might be something they can eventually also do, but it’s not the shortest path to follow to get better at species ID than they are today.
That works fine for organisms that have field guides like flowers, birds, butterflies, etc. For a lot of other groups, that information is harder to cite.
I vaguely recall staff talking about thinking about adding something like this, but I don’t remember where and I don’t see a request with an “under review” tag.
One of the ideas from one of their annual retreat/prioritizing/brainstorming sessions (2019): “Investigate ways to capture comments and ID remarks that are useful for making identifications and including them on the taxon page and in identification tools.”
Completely agree. I will copy and paste a post I made that was not approved because your topic already existed:
Integrating species identification features in their pages.
I often find myself wondering if a flower is from a species or another; if the different pictures of a bird are just because it’s male versus female; etc. Further, we are all humans and benefit from details about other senses other than images: not only that, but some people with low vision rely on touch and smell even more and this feature would allow them to participate and identify things better.
URLs (aka web addresses) of any pages, if relevant:
Description of need:
Need:
A way to add varied information and descriptions to a species’ page.
Why?:
Visual appearance is not the only way we identify things. Smell, touch, sound and specific notes about distinctive features are not only complementary but sometimes essential characteristics to identify things. Imagine also that some of us have less good vision, but we know the plant is very spiky. We can’t figure out by the pictures alone if the plant is spiky or not, but when we touch it we know it’s species A and not species B because of that.
They all look pretty similar, for her non-specialist perspective. She wonders: what are the distinguishing features of these poppies? Is it the petal shape? Number of petals? Something to do with the leaves? How do they look like at early life stage, and how do they look like at the end of the life stage (because the pictures they show are mostly of the most bright moment of the flower).
User Journey 2:
Alex takes a picture of a bird but also records audio.
He doesn’t know the distinguishing features of the bird, compared to other similar species within the fame family/taxon. He doesn’t know what the bird’s call is. He doesn’t know if the pictures look different just because one is male and one is female.
Feature request details:
Sam needs that the species page provides the following information:
Distinguishing features: what makes this poppy different from other poppies? What colors does it have, how many petals does it have, what’s the texture of the petals, what’s the shape of the leaves, etc. (smell as well - some plants have specific smells)
Distinguishing features must be discriminated by season/life stage.
Alex needs, in the species page:
Distinguishing features: Described in text.;
Distinguishing sounds features: bird sounds, including call, courting songs, etc.
Distinguishing features between male and female, young and old birds: in text and with pictures.
Honestly, I would love to know if this is still considered. If it is, it would be also great if any update could be given on it’s priority - in case the staff are interested but simply are focused on other developments (e.g. new mobile app) which would be understandable.
Please see comments from the community for details on how they want it implemented. I’ll copy the response I left on blog post in here as well:
I support the idea of implementing a user-maintained wiki for having identification and other info easy to see to all people on the site, that perhaps could be tied to the taxon pages themselves.
There were already ‘Guides’ but the feature is no longer supported/being developed. It’s not well integrated or well visible on the site either which is a big issue. https://www.inaturalist.org/guides
Wikipedia wouldn’t be great for this either, as it has a completely different purpose and writing style.
…the Guides are maintained by each person separately, and once again hard to notice, which doesn’t fit exactly with the purpose of a wiki.
I have a string of URLs bookmarked for iNat. I include the number of pages waiting in that URL name. Some of us are working thru the backlog from the CNC ‘winner’ 3.5K Disagreements paired with 3% RG