Very good point, I know a few very knowledgeable identifiers who are professional in the field irl but have <1k identifications on iNat.
Will there be a way to mark certain identifiers as “trusted//knowledgeable”?
3 Likes
I think they’re still working this out, but one thing that I love about iNat, is that no one cares what institution you work at. No one cares what books you’ve published . . .
The only questions in this context are, “Have you made a lot of comments on iNat? Have your comments provided helpful tips for ID?”
A pure meritocracy.
5 Likes
If you’re willing to check out the demo and share your thoughts, that would be really helpful. We’re here to kick the tires of it, so to speak.
Like @sethshively and @dlevitis have said, the actual comments from IDers are very useful on their own! Having a few key comments featured on a taxon page would be very helpful, particularly ones that touch on how to distinguish that particular taxon from similar-looking ones. (Or even whether you can’t, e. g. “in X region you will need a microscope and a key to tell this species apart from others, so if you want to be on the safe side just ID it as Genus”, or “if you are outside [native range] you are probably actually looking at the cosmopolitan Other species”.)
However, the original comments are automatically hidden in the current demo. Many if not most people will not click the drop-down; they’ll assume, rightly or wrongly, that the LLM summary is accurate. (We can predict this because of how many people don’t even scroll past Google’s AI summary or click on the source link to see if it actually does say that.)
Various people (including you!) who are deeply familiar with certain taxa have pointed out where the summaries are subtly misleading in ways only an already knowledgeable person can spot. If you need an expert to double-check everything anyway, why not just quote them directly in the first place?
While I appreciate your request for feedback, I am not particularly interested in helping improve this demo because I think generative AI is fundamentally unsuited to this purpose. An LLM like Gemini is not designed to be accurate, or even really to summarize. It is designed to produce text that looks like it’s accurate or looks like a summary. That is good enough for Google, because Google’s business strategy is essentially to become Cymothoa exigua but for the Internet. (No need to click through. The isopod will taste it for you.) However, I think iNaturalist should be aiming for a higher goal: actually being accurate.
I have some loose ends to wrap up here, but I plan to delete my account by the end of the year. I’ll check back in 2-3 years.
10 Likes
I agree with this and will add, I rarely see comments on how to ID a species from people who don’t know how to ID. Even known amateurs or autodidacts seem to use correct identification characteristics most of the time. People who don’t know those traits don’t really add those types of comments OR they seem to copy comments from those who do seem to know. All of this is to say, I rarely see incorrect identification tips, though on occasion I see comments of things that aren’t really diagnostic. I think the volume of useful comments vastly outweighs those that are off topic or incorrect. So if the LLM is looking for the comment with highest probability of being correct because they’re the most used, it will settle on those that ARE correct. This has already changed how I, personally, identify in that I am more likely to repeat the same comment so as to help the model settle on accurate comments. It’s also changed how I add comments in that I am starting to point out the diagnostic characteristics for the higher level taxonomic rank, and not just the species-level endpoints.
5 Likes
As the top Lucilia “expert” for iNat in North and South America, I would vouch for the AI summaries being more coherent and helpful than the original comments themselves, for someone who doesn’t have time to read thru >3 different comments that all approach a single character from different points of view.
Sure, there’s a bit of room for improvement, but this approach still seems much more practical than making a wiki from scratch. The AI summaries do a particularly good job of providing context and synthesis. The ‘ID Summaries Feedback’ tool will probably be adequate, I hope.
A lot of the complains so far seem to be things that I expect to get better once the demo is trained on a larger set of comments.
20 Likes
Save us, please, from some of the self-identified experts on iNat! I think the creation of ID materials should be done by recognized experts, not self-promoters or AI. If need be, I think we’d be better off having staff or volunteers research field marks / ID traits. Maybe we could have iIdent, where people post/list helpful fieldmarks for community agreement.
2 Likes
This is horrible news and exactly what I and so many others implored you to stop before it was too late. Is there a way to completely opt out of our identifications ever being used for this insidious project, or do we need to delete our accounts in order to prevent our identification expertise from being fed to this AI abomination?
3 Likes
this. highly considering deleting my account after all my comments, it’s frankly violating and disgusting to have our passion sifted through with seemingly no implementation of an opt out , then digested into a usually incoherent blob incapable of ever understanding the morphology or anatomy being referenced, so bureaucrats can masquerade this demo as a breakthrough for lowering the boundary to identify (a wiki and an actually improved cv would do the trick..), while in reality it’s just for the grant money.
3 Likes
As it currently exists the demo is explicitly opt-in only, and trained off 30 identifiers who volunteered. I personally would be quite upset and shocked if they ever made it so there was no way of opting out of this.
13 Likes
Fair, but I have seen far too many instances of unwanted products rolled out in small batches that were later mandatory for all users, so I am not optimistic.
1 Like
One thing that is still perplexing to me regardless if one loves or hates this. It seems very very few people are engaging with the “ID Summaries Feedback”.
If you love this, it is a benefit to the devs and iNaturalist team to see how many people think the explanations are helpful, correct, etc, and if somethings wrong. Making that heard so it can potentially be improved.
If you dislike this, isn’t it a good idea to show potentially why by highlighting the mistakes?
To me it seems the only downside to giving feedback with the votes is if you really dislike it and want nothing to do with it, or the time needed to go through them. But perhaps I’m missing something.
Although, am I to believe nobody here in this post is able to verify if the AI summary of Eastern Poison Ivy is correct? Like one of the most common and publicly known plants in the Eastern US? Will note this demo seems very US centric at the moment also.
6 Likes
i get that, and i promise you i am as sick as anyone of slop taking over websites i care about, but i think there is at least as much reason to expect things to go the other way. from my understanding google gave $1 million of no strings attached money on more or less the sole condition that they make some prototype that uses genAI. i would be feeling very differently if this was cooked up internally by the team because they were certain the future of iNat would be all about talking to LLMs or whatever. maybe i have a cynical view of grants in general but them delivering the prototype stipulated by the grant does not make me too worried that the team has fully bought in to genAI and is going to start rapidly expanding this in intrusive ways.
if that ends up being wrong and they start filling the actual website with genAI hallucinations and start training models on peoples comments without even allowing them the option of opting out i will happily join you in raising a fuss! but until then i’m going to just wait and see, especially when there’s direct precedent for them making a prototype for a grant like this and then leaving it completely siloed off from the main site in its original state.
10 Likes
in my opinion, that feedback mechanism design in the demo is suboptimal, and i don’t see how the feedback collected through it can be used practically, even in some sort of manual analysis. i’m not a hater of the overall project – i suggested doing something like this a while back – but to me, better feedback can be provided by other means.
7 Likes
Going to need AI to summarize this topic at the end.
I do think this concept will be of great benefit in collating all this knowledge. What may be a nice addition is a glossary of terminology used in the summaries.
For example pronotum was used in the lady beetle sample. I had no idea what it referred to until explained.
3 Likes
If I link to an article containing an identification key in a comment, will the AI read it and pick out the relevant details?
1 Like
We need to start from - is this observer / identifier interested in learning to ID this. Or do we want more basic skills to move the obs along one step (passing the baton) so the assorted taxon specialists can see it. A good comment from a taxon specialist - to take me to THAT obs - and then I can weigh up IF I can use the ID help, or if the obs is worth their time and effort for me to @mention them.
Maybe we need for example - ID 101 for non-plants (if I have to look up all the words then that doesn’t work for me), and ID 201 for plants for me. We need a mouseover prompt - what are halteres - where are they? I know they are ‘part of a fly’ a fly field mark.
2 Likes
This is a good idea in theory, but hard to implement for certain taxa. For example, in reptiles, each scale has a name so that one can encounter nasals, post nasals, prefrontals, frontals, frontoparietals, parietals, interparietals, occipitals, nuchals all occur on the head (in this case, in order from front to back). Okay, fine, but here’s where it gets tricky. Not every taxon has every scale meaning some groups might not have the interparietals or prefrontals and knowing whether that scale occurs in that group can be part of the identification process. This can make identifying even tougher because there’s no one resource available to show each of these in all taxa. It gets worse, in some cases, the name given to a scale in one group is different in another group. So in order for this to work, there would need to be a resource to reference, and those are usually buried in the original taxonomic descriptions. It’s a good idea, but the context would make it harder (not impossible, just harder) to implement.
2 Likes
That is where I would split it. Generalists like me will point those obs at herpers - and they know which scale is which. Or are learning them. (and in that case - don’t @mention unless there are 2 or 3 focused views in the obs, which might show the needed field marks)
2 Likes
I also would be surprised if that happened, but it would be very helpful if iNaturalist staff could explicitly confirm that this project will continue to be opt-in, or, if not, whether there will be a way to opt out.
To emphasize, I know the current demo is opt-in. I would like to know if it will continue to be if the project expands.
Without that reassurance, I think it’s fair to consider it a possibility, even if it’s a very small one. I realize this is in the very early stages, but this is such a basic consideration that they surely must have discussed it?
I haven’t been checking in on this conversation regularly, so if they have said something and I’ve just missed it, someone please let me know!
8 Likes
The demo is undoubtably useful as a domain-specific search tool for finding relevant in comments. It might also be useful as a way to seed a centralised repository of guidance materials - but without wiki-like editing tools, it’s hard to work up much enthusiasm for contributing to it.
The current summaries seem rather cosmetic and the feedback options blandly ambiguous. I cannot see why I would I want to spend time reviewing automated output over which community members have so little control. If something’s not worth doing, it’s not worth doing well. The original comments are often too brief to be merit summarising, anyway, and are generally better viewed within the context of the observations they relate to. Automated harvesting of comments is fine, but any summarisation should be the starting point for creating guidance materials, not the end point.
A major problem with the structure of the current system is that it’s fundamentally non-collaborative at the point of entry. Commenters work within the isolated context of an observation, so there’s enormous potential for duplication of effort, and no clear indication of how, if, or when a given comment will be utilised. This model seems backwards and unnecessarily inefficient, when compared with something like a wiki, where contributors collaborate directly on a centralised resource.
14 Likes