Exploring New Ways to Learn from iNaturalist’s Community Expertise

[danlharp]

@tiwane

Hi everyone, I just added an update to the first post in this thread about the ID Summaries Demo. In summary, you can now see and explore the first-ever version of the ID Summaries Demo for yourself. Thanks again for all your ideas and feedback. We’ll continue exploring how to make it easier to find and access identification info here on iNat.

Once again, thank you to iNat staff for working on ways to help iNat users make better observations.

Some general feedback (that doesn’t fit into the feedback reporting in the demo)… As constructive criticism, I’ll compare the demo to two websites I use regularly to help me observe better: Native Plant Trust “Go Botany” online key for vascular plants, and Cornell Lab’s “All About Birds” website.

“Go Botany”: Technical terms are hyperlinked to definitions whereas in the demo terms like “leaf sinus” and “globose” may be confusing. Each species account has photos taken by botanists specifically to show important features (it makes a difference when botanists take the photos!). The prose is technical, but not dreary like the demo.

“All About Birds”: I compared Mallard in the demo vs. “All About Birds.” While I’m not much of a birder (only 300 on my life list), I’ve done enough birding to make it feel like the demo spending so much time on bill shape feels a bit…I don’t know, strange. “All About Birds” focuses on what my serious birding friends would focus on: structure and feather patterns (but not bill shape). The photo tips in the demo feel off, too.

I had hopes for this project, but so far I’m underwhelmed. Maybe some of that comes from feeling that the demo is kind of unfocused – it’s trying to be a field guide, and a technical reference, and a series of photo tips, and so far I feel it’s doing none of those things especially well. I also feel the demo doesn’t have a clear target audience – with technical terms like “globose” used under Acer rubrum, this is not exactly beginner-friendly – yet someone who’s familiar with the term “globose” is probably going to have a dichotomous key, or at least a field guide.

Then I started thinking – would I use this to help me better observe taxa about which I know nothing? Given that it seems a little weird for two taxa about which I know a little bit (Aves and Angiospermae), I’d be kinda reluctant to trust it for anything else.

Please don’t take this as completely negative. I can see how useful this could be, especially for geographic regions where there aren’t great resources – and also for human languages in which there aren’t a lot of resources. I can also see how using a LLM seems to offer an inexpensive way to create some kind of field guide / observation guide.

At this point, though, I’m not convinced that using a LLM model trained off ID comments is the best approach. Elsewhere on the Forum, iNatters have suggested a wiki approach – which I initially felt dismissive of, since a wiki would take staff time to supervise. But when I see how much staff time must have gone into creating this demo, with fairly mediocre results… when I think about the large number of organisms that won’t have enough IDer comments to make it into something like this (most lichens, most of Bryophyta s.l., most of Rhodophyta / Chlorophyta / Phaeophyceae, etc., to name a few of the taxa that bog me down right now)… when I see that already people are finding errors that need to be corrected… well, a wiki sounds more and more feasible. Especially given the user base of iNat who are already committed to putting in long hours.

In sum… great impulse behind this project, I’m so appreciative of the work that’s gone into it… but unclear audience, unfocused product, and there appear to be other possibilities that might work better. And I’m so sorry if this sounds harsh!

[sethshively]

I believe others have already said this, but instead of making AI-generated summaries from comments explaining ID, why not just have a feature that shows said comments verbatim? It is very unlikely that AI-generated rehashes of people’s comments will ever be more coherent or helpful than the original comments themselves. If you have to use some form of AI for something (because apparently that’s mandatory for literally every software product nowadays), why not just use a selection algorithm to pick out the comments it deems most relevant and show them as is?

[thebeachcomber]

@sethshively

why not just have a feature that shows said comments verbatim?

this is also included in the current demo, if you click the ‘iNaturalist User Comment References’ text. I understand you’re asking for that instead of the summaries, rather than in addition to, but wanted to highlight them just in case you hadn’t seen these

[escol]

@zoology123

Some taxonomic groups on iNaturalist have fewer than 3 large active identifiers.

This also brings up, who wants to write something 30-50ish times? Even if you use copy and paste, how is this ID summary going to handle 30 identical comments? Write it almost word for word? I still advocate for the direct addition of identification information to taxa from knowledgeable users whether it is in the form of a wiki, guide, a new tab on the species page, or some other method.

Agree here… Some taxa I only identified 1-3 times (and so only identified by one person) but yet personally I would consider enough to include information about the species on let’s say a wiki.

@zoology123

You could write identification information for species with no observations on iNaturalist for the day they are observed.

^ This as well, I believe this is a good idea. Current approach would suffer with lesser known regions and lesser known species.

@zoology123

Well, I don’t think any Chironomids will probably ever get in then. While it sounds good on paper and even in practice it is good to have multiple sources (different people commenting information)

Could be replaced just by one (or more) source being a scientific paper, or a comment referring to one saying directly what’s in the paper.

[zoology123]

@escol

Could be replaced just by one (or more) source being a scientific paper, or a comment referring to one saying directly what’s in the paper.

Honestly, most of my detailed comments are more like a bibliography.
Example snippet from https://www.inaturalist.org/observations/313418124

Not exactly correct scientific citation, but here are most of the sources I used.

Fittkau, 1962, DIE TANYPODINAE

Murray, 1976, Thienemannimyia pseudocarnea n.sp., a palaearctic species of the Tanypodinae (Diptera: Chironomidae) https://brill.com/view/journals/ise/7/3/article-p191_5.xml?language=en

Roback, 1971, Adults of the Subfamily Tanypodinae (-Pelopinae) in North America (Diptera) https://www.google.com/books/edition/Adults_of_the_Subfamily_Tanypodinae_Pelo/lYTr8SHK1u0C?hl=en&gbpv=0

Coe, Freeman, Mattingly, 1950, HANDBOOKS FOR THE IDENTIFICATION OF BRITISH INSECTS https://www.royensoc.co.uk/wp-content/uploads/2022/01/Vol09_Part02_3_Chironomidae.pdf
Page 6 b is wing illustration for R. eximia

https://v3.boldsystems.org/index.php/TaxBrowser_Taxonpage?taxid=102878

[zoology123]

Because these descriptions are from comments. There is an incredibly large flaw in that the comments almost never are fully describing the organism. Usually comments only focus on what is the bare minimum needed to ID. Remember, any LLM only “knows” what it knows. If it is not provided information from comments on an insects leg pattern, wing pattern, color, size, antenna, etc. It will be unable to include those in the description.

So for an example, here is a full description of Rheopelopia eximia.

R. eximia description from Fittkau, 1962.
Translated from a German scanned book.

“Note: Since its description, eximia has not been found again. EDWARDS only knew 2 females from England. It seems to be a rare species. The first male from the Black Forest is now available (leg. Wülker). The affiliation with eximia is beyond doubt due to the characteristic wing coloring. In general morphology, Rh. eximia is largely similar to Rh. maculipennis, but shows distinct features in the shape of the Coxitlobus, in the AR and especially in the body coloration. Both species remind us of the species pairs in Thienemannimyia (lentiginosa-laeta and festiva-carnea) in their differences. It is plausible to suspect a strong cold stenothermy in the darker colored species as well. Its occurrence supports this. Rh. eximia could be the more primitive form compared to maculipennis, as the palps are still brown here and tibial rings are formed.”

“Coloration: Mesonotal bands black-brown, as well as the postnotum, mesopleura, and mesosternum; the latter is light just below the episternal suture. Praescutellar field slightly brownish. Scutellum light. Palps brown. Wings with characteristic markings (cf. EDWARDS 1929, Plate XVII, Fig. 2), based on both membrane and macrotrichial coloration. Cross veins dark, especially in their proximal area a dark spot. A dark field in the anterior part. Almost the entire distal wing half is dark with various light, round fields. One field under r4+5 distal to r3, a very small one at the end of c, a slightly larger one above it just before its end, and each one larger at the wing edge in M-CU. Dark spots on r2 and at the junction of r3 as well as at the end of r4+5. Halteres whitish. Legs with a pre-apical ring on the femur and another ring before the base of the tibia. The last tarsal segments darker than the others. Abdominal segments II-VI with an oral brown band that covers about one third of the tergites. Segments VII and VIII are entirely dark except for a narrow oral stripe. Hypopygium brownish.”

How do you describe a complicated wing pattern in text form?
Example.
“Cross veins dark, especially in their proximal area a dark spot. A dark field in the anterior part. Almost the entire distal wing half is dark with various light, round fields. One field under r4+5 distal to r3, a very small one at the end of c, a slightly larger one above it just before its end, and each one larger at the wing edge in M-Cu. Dark spots on r2 and at the junction of r3 as well as at the end of r4+5.”

To the average person you may as well be speaking a different language. This text alone would probably be useless to almost everybody that sees it. There is an alternative though. One can just look at the figure.

In the current stage of this demo, it seems almost infeasible to describe the pattern of this wing without a figure, or a figure with a list of wing veins where you would have to map out the pattern in your head looking at the wing vein diagram shown below.

Source https://www.royensoc.co.uk/wp-content/uploads/2022/01/Vol09_Part02_3_Chironomidae.pdf

Thinking back that an LLM only knows what it knows. If I did not copy paste its full description. The LLM would only see.

Patterened wing eliminates
T. vitellina
T. geijskesi

Tibia base with dark ring eliminates
T. woodi
T. carnea
T. fusciceps
T. festiva
T. northumbrica
T. pseudocarnea
R. maculipennis
R. ornata

This leaves
T. laeta
T. lentiginosa
R. eximia

T. laeta eliminated.
“Tergite II only laterally with a small oral spot at the oral edge on each side.”

T. laeta, and T. lentiginosa both have different wing patterns to R. eximia.

This comment alone is almost useless for this LLM. It barely describes color, wing pattern, size, shape, wing venation, or really anything besides the bare minimum used to separate from the other species.

This is why I so strongly advocate for a wiki. Where one can add all the details they want even if it’s not likely to be the most useful. You can really try and get an understanding of an organism from it’s full description and that helps compare it with other similar species if you have their descriptions as well.

You can’t expect every identifier to list all these details for an LLM for each comment, like this beetle is dark brown as is 10,000 others. This means the LLM will always be lacking details on the species, potentially even obvious ones.

[martin487]

Great idea to harvest the valuable assets of ID Tips from Comments using AI.

Understand the concerns though in the comments above ensuring the final output is accurate and non-repetitive.

Would be great to have final curated ID Tips (including tips for photos to get an accurate IDs etc) on a new tab on the View species screens accessible from Upload and Identify screens, like these

[DianaStuder]

I would like - ID tips - to be a tab on a taxon page. How do I ID … versus whatever the confusions are for that taxon. Wouldn’t have to ‘remember where to find it again’ which is what I have now with an abandoned pile of bookmarked URLs. And it would be up front visible if you make the initial effort to go to the taxon page for more info.

Here is a bird https://www.inaturalist.org/observations/276973590 Which sp ? Off-topic would like an expand and collapse option on obs - so we could immediately see a short tidy list of who is on this long thread of IDs and comments. Have I already asked … or have they already weighed in ??

@dlevitis

Please do roll out a tool that lets us easily find IDing advice from top IDers

Actually that is what I want - but then searchable by location too. Who is the informed (still active, and helpful) taxon specialist, that I can @mention for this ? And in time my learning curve will help me to narrow down - jumping legs = Orthoptera, but this one, is a katydid. And that ‘ant’ is the nymph of an ant-mimic katydid, while this one is an ant-mimic spider.

[pisum]

@gcwarbler

“Photo Tips”

i’m not sure if i would have noticed these if gcwarbler hadn’t pointed these out, but i think this is probably among the top 2 most interesting things in the demo, not because it’s particularly useful right now, but because it shows where iNat staff might be thinking of going with all of this in the long run, since it’s another baby step toward to achieving real-time guided augmented reality observation / identification and also to linking a language model with the vision model.

i looked a bit at the demo, but besides the Photo Tips, it looks more or less like the screenshot of the demo presented back in October, and all my comments from then more or less still apply.

[GeolTel]

For the few plants in the demo that I’m familiar with (C. sepium, Erodiums etc.) in my tiny part of the world, I find the “id summaries” neither really helpful nor reliable. Almost tautological at times (statement like “the shape of […] distinguishes it from species with a different shape”). Are the “ID summaries” designed by and intended for Californians? I see no mention of the area of applicability of these “tips”. Nor a way to provide feedback on a particular item.

[DianaStuder]

@GeolTel

Nor a way to provide feedback

click on the right sidebar - where it says - ID Summaries Feedback.

Since it is outside of iNat the demo is up, while the site is down.

I imagine this first demo is California based. Not seeing any biodiversity I recognise - beyond flower / insect.

[GeolTel]

Oh, probably a bug on the demo website, then: when clicking on ‘ID Summaries Feedback’ there was a single feedback form for the whole list of ‘ID Summaries’. Now there’s one feedback form in front of each ‘summary’ item. Sorry for the noise.

[nancylightfoot]

I just looked at the sample for my area of expertise. I thought it was really excellent in some ways, but there were a few decisions that made it less useful than it could be for me.

First, thank you SO MUCH for featuring a female bird in a sexually dimorphic species. I was looking for photos of a female on All About Birds recently and found one female bird among eight or so photos. Note that this will compound a critical bias that has important conservation implications—if we don’t show females, we won’t recognize and photograph female birds either. And often the number of females is limiting in terms of reproduction / health of the species.

The sample was for Mallard, and there were a few key features missing. First, if the bird has white tail feathers, that is a definitive field mark for female Mallard vs. other brown ducks. Domestic Mallard is often all white, but per Kevin McGowan at the Cornell Lab, any white in the wings indicates Domestic Mallard as well. Identification of key field marks by an expert birder would be much more helpful than a string of comments, which I found distracting and unhelpful. Other helpful examples: The best fieldmark for a Red-tailed Hawk in flight is a dark patagial bar–a bar on the leading edge of the wing from the neck to the first crook in the wing. A rusty or even pinkish tail is definitive for a Red-tailed Hawk in North America. Distinguishing between Greater and Lesser Yellowlegs can be difficult in some angles. If the bird has a upturned look to the bill that indicates Greater Yellowlegs. Look for a black “shoulder” (leading edge to the wing) for nonbreeding Sanderlings. If visible, that is an excellent fieldmark. Etc.

The compare to similar species feature in the Cornell Lab’s All About Bird account is one of the most useful ID aids I’ve encountered. I think a few comments about eliminating similar species would be super helpful, although the photos are really important as well. If you’re really committed to the string of comments vs. curated lists of critical field marks, I would appreciate a button to just bring that up if you’re interested / opt out.

[roadside_rambler]

I just looked at a species I’m highly familiar with, Common Pawpaw, and the AI summary did a really lousy job on the leaf discription.

It somehow turned an observer’s comment that:

“There are numerous smaller and quite isolated plants in the general vicinity (within a few hundreds of yards) that were easily spotted by their large yellow leaves.”

Into an exaggerated and misleading blanket statement that:

“In autumn, the foliage turns a striking yellow”

I’ve no doubt that the first statement was true, in that place on that day, but it said nothing about the yellow being “striking” nor did it make any claim that this was always true, or distinctive enough to be used in identification. Where I live, pawpaw leaves don’t turn yellow in autumn at all - they actually stand out in autumn by remaining green long after everything around them turns bright yellow (mostly tulip poplar, shaded sugar maple, and spicebush) then depending on how quickly the weather turns, either fall to the ground green, or turn a watery tan color a day or two before dropping. I also think it’s kind of weird that a program meant to use simple language turned “leaves” into “foliage.”

Then it went on to turn a helpful but not ID related comment on an observation of a young, planted, wilted pawpaw in someone’s yard:

“Give this tree a lot of water :-) The leaves droop like that when they get water stressed.”

Into an identification feature:

“…and leaves may droop downwards when the plant is water-stressed.”

Well, yes, “may” but that’s true of almost any broad-leaved plant. I’ve yet to see a wilted pawpaw in the wild, even during times of drought. They probably grow in moist shady areas for a reason.

I’m not sure I have the patience to check out the demo for any other species, after how bad this one was. I will just wait and see for now, but if this ends up forced on us I will definitely be leaving iNat for good. It’s just too depressing to see perfectly good comments mangled into AI slop.

(And yes, I used the built in feedback, but it felt inadequate since there was no way to explain what was wrong with the summary.)

[yayemaster]

The Ilex verticillata on the demo is actually Prunus serotina. How can that be removed to avoid misinformation? The feedback only applies to the comments below.

[escol]

Yes, I agree and it’s not always possible to just figure out by one source and just quoting one. I do it the same way as you I suppose - providing checklist, leaving comments comparing different features, attaching descriptions along the way, or keys. Each step hopefully with a paper attached to back it up, but sometimes I try to figure out additional details from photos of lesser known taxa e.g. for something that wasn’t obvious in existing key - something that would constitute for a ‘tip’ but only having experience identifying a species to back it up rather than a specific source.

Either way (and to not go off-topic too much), I’m hoping to see how this turns out outside of the demo.

[holyegg]

I watched some but not all of the webinar, so apologies if this has already been answered.

How does this tool react/adapt to taxon changes? The premise seems to be that ID tips are static, but they may change dramatically as taxa are lumped, split, swapped, etc. An ID tip from the site on Arceuthobium campylopodum from 2016 has no utility now, because the species concept in use in 2016 differs dramatically from the circumscription we use now on the site. This is further complicated by the fact that when a taxon is split, it (with the same taxon ID #) is often used as an output.

One could use the upvote/downvote system to mark ID tips that are now obsolete. At that point, though, aren’t we essentially curating a wiki? If an IDer or curator for a taxon does not actively engage with this tool, and there are taxon changes, then there is a lot of potential for outdated information to propagate incorrect IDs.

[tiwane]

That’s been fixed.

[chris_lamb]

After spending time the past couple of years figuring out how iNat works, I think the tool has great potential as a starting place for learning. With tweaks/edits/additions/improvements by iNat people, it can be extremely useful, especially to new iNat users or those learning a new taxon.

Of course iNat experts can do a better job than the AI, but the time it would take them to do it is a different matter. So if the tool can jump start the process, iNat people can expand on it, and fix anything the AI doesn’t get right. A symbol so users know if the section has been improved, or vetted beyond the AI would be helpful.

Looking at the Mallard example, an easy or “beginner” bird, until you come across a female vs. American Black Duck, domestic / hybrids. The examples mention some of these points, but additional explanations are needed. A great start, and the photo tips are helpful.

As a generalist who has learned a lot, yet only a tiny fraction, tools like this would speed up the process. Looking forward to seeing Common Dandelion and Red-seeded Dandelion added to the set.

[upupa-epops]

@chris_lamb

Looking at the Mallard example, an easy or “beginner” bird, until you come across a female vs. American Black Duck, domestic / hybrids. The examples mention some of these points, but additional explanations are needed. A great start, and the photo tips are helpful.

True, the comments would probably need to be more nuanced and detailed in order to be helpful at levels useful both for newer observers struggling with more basic ID (e.g. distinguishing female Mallards from teal, shovelers, wigeons) as well as for more advanced ID with similar species and their hybrids (e.g. Mexican, Mottled, American and Pacific Black Ducks).

@chris_lamb

Looking forward to seeing Common Dandelion and Red-seeded Dandelion added to the set.

Oh boy, that would be interesting given the ID comments will have a lot of direct contradictions.