But that is something they wanna achieve with the new image to text algorithm they are testing.
Calling it “image to text” isn’t an accurate description of what they are actually doing, and doesn’t give a realistic sense of what it is actually capable of. Really it’s more the opposite of that - it’s a text to image classifier.
It’s not dissecting images and describing their component parts, it’s classifying their similarity to the keywords in a human captioned training set - so when you search for “bird on a car”, it’s looking for the images with the best match to the ones trained with those caption key words.
And it’s still involving humans in the ongoing training to tell it when it returned something like what you asked for, and when it got that hopelessly wrong.
I don’t want to rain on your enthusiasm, but being realistic about how this works and what it is actually capable of is the best way to find genuinely good uses for it.
So maschine learning could quite well work on figuring out what a wing and what a thorax is.
That is one thing it could be trained to do, but if we’ve learned anything about “machine learning” and (more accurately) search and optimisation since circa the 50’s when some people asserted that “an electronic brain which could surpass Man” was just a decade and a few million dollars away - it’s that approaches which just try to mimic the way a human would do this are unlikely to work as well as those which better exploit things that the machine is actually good at doing …
So that might be something they can eventually also do, but it’s not the shortest path to follow to get better at species ID than they are today.
That works fine for organisms that have field guides like flowers, birds, butterflies, etc. For a lot of other groups, that information is harder to cite.
I vaguely recall staff talking about thinking about adding something like this, but I don’t remember where and I don’t see a request with an “under review” tag.
One of the ideas from one of their annual retreat/prioritizing/brainstorming sessions (2019): “Investigate ways to capture comments and ID remarks that are useful for making identifications and including them on the taxon page and in identification tools.”
Completely agree. I will copy and paste a post I made that was not approved because your topic already existed:
Integrating species identification features in their pages.
I often find myself wondering if a flower is from a species or another; if the different pictures of a bird are just because it’s male versus female; etc. Further, we are all humans and benefit from details about other senses other than images: not only that, but some people with low vision rely on touch and smell even more and this feature would allow them to participate and identify things better.
URLs (aka web addresses) of any pages, if relevant:
Description of need:
Need:
A way to add varied information and descriptions to a species’ page.
Why?:
Visual appearance is not the only way we identify things. Smell, touch, sound and specific notes about distinctive features are not only complementary but sometimes essential characteristics to identify things. Imagine also that some of us have less good vision, but we know the plant is very spiky. We can’t figure out by the pictures alone if the plant is spiky or not, but when we touch it we know it’s species A and not species B because of that.
They all look pretty similar, for her non-specialist perspective. She wonders: what are the distinguishing features of these poppies? Is it the petal shape? Number of petals? Something to do with the leaves? How do they look like at early life stage, and how do they look like at the end of the life stage (because the pictures they show are mostly of the most bright moment of the flower).
User Journey 2:
Alex takes a picture of a bird but also records audio.
He doesn’t know the distinguishing features of the bird, compared to other similar species within the fame family/taxon. He doesn’t know what the bird’s call is. He doesn’t know if the pictures look different just because one is male and one is female.
Feature request details:
Sam needs that the species page provides the following information:
Distinguishing features: what makes this poppy different from other poppies? What colors does it have, how many petals does it have, what’s the texture of the petals, what’s the shape of the leaves, etc. (smell as well - some plants have specific smells)
Distinguishing features must be discriminated by season/life stage.
Alex needs, in the species page:
Distinguishing features: Described in text.;
Distinguishing sounds features: bird sounds, including call, courting songs, etc.
Distinguishing features between male and female, young and old birds: in text and with pictures.
Honestly, I would love to know if this is still considered. If it is, it would be also great if any update could be given on it’s priority - in case the staff are interested but simply are focused on other developments (e.g. new mobile app) which would be understandable.
Please see comments from the community for details on how they want it implemented. I’ll copy the response I left on blog post in here as well:
I support the idea of implementing a user-maintained wiki for having identification and other info easy to see to all people on the site, that perhaps could be tied to the taxon pages themselves.
There were already ‘Guides’ but the feature is no longer supported/being developed. It’s not well integrated or well visible on the site either which is a big issue. https://www.inaturalist.org/guides
Wikipedia wouldn’t be great for this either, as it has a completely different purpose and writing style.
…the Guides are maintained by each person separately, and once again hard to notice, which doesn’t fit exactly with the purpose of a wiki.
I have a string of URLs bookmarked for iNat. I include the number of pages waiting in that URL name. Some of us are working thru the backlog from the CNC ‘winner’ 3.5K Disagreements paired with 3% RG
Not great that its closed. This seems to be overwhelmingly what people are asking for rather than the blog post with ‘generative AI’, or however their plan is.
It looks like iNat still has a similar goal on which they ended this discussion. Now it’s only the matter on how they want it implemented, from what we see they decided to try out aggregating information from comments and train a (demo) generative model on it, so it can explain a choice of certain species (along CV but different approach?).
But then the majority of people don’t like this approach, and many just want the wiki thing.
I personally believe having information open like this can have a greater impact than just facilitating identification (which is if we choose to do it through AI tips). Maybe more people could start learning less popular or less known groups, etc.
I agree, About tab is also a good fit. The simpler the better (so, to some degree they could even be combined, if given a good layout).
Can also allow to switch between the info from iNat and Wikipedia (which is currently used in About section)?
Sidenote rant: Sometimes I feel like I’d rather select ‘Similar species’ myself because it doesn’t always fit the ID-ing experience. And sometimes there are species that are not even similar, but rather it’s what CV gave to users - on accident, due to not being perfect and users not even bothering to look at any option at all to compare (just selecting the first shown CV ID).
I always appreciate your carpet beetle information. I hope this tool does end up implemented in some way, I was thinking about it today. So many have identification knowledge to share, its such valuable information but so hard to find. All of the species requirements and information compiled into inaturalist would be revolutionary, especially for entomology. I’d be happy to help make diagrams for these info pages of any species (esp true hoppers) if they’re ever added.
I also feel the same for similar species, there’s so many that I could list as similar to eachother if misidentification wasn’t the requirement to create those lists. having those about pages would help explain the differences between some of the similar species.
Ultimately the often suggested Wikipedia is not good (nor any other existing options). People suggest implementing a new resource focused highly on the ID guide aspect, and it’s still not clear what iNat really wants or doesn’t want to implement (although maybe we will know soon..?).
I want to be able to write what is needed to identify certain things and that writing show up when using the CV. I’m tired of people using the CV and not realizing something needs genitalia, or x to ID. Everyday pushing a bunch of observations to higher levels.
You can’t fault people for not knowing obscure knowledge.
I agree, this is why it works better integrated into iNat itself (rather than a separate resource). And the genAI feature, even if it finds such knowledge on X taxon and writes it somewhere, should allow to edit the result.
Then I see no reason they just couldn’t call it “we’re gonna add an identification guide and will populate the identification articles + CV captions from user comments, for a quick start (if they want their comments to be added here in the first place - opt in, or opt out)”. Sounds much nicer and more promising, right?
For the CV itself, maybe capping a species to a certain rank would also be helpful in such situations. e.g. several species in CV but restriction for the genus applies so it can’t suggest anything further than such genus. I’m sure this idea surfaced somewhere on the forum in one form or the other.
We’ll still have the kind of people that don’t look at what IDs they’re even adding, and so possibly ignoring such captions in CV in future. But having new useful features like this should solve the part of issue.