Stance on AI assistants to write responses to identifications?

General
, ,
1

I am starting to find the following exchanges on Inat:

  • Uploader very excited with a unique observation.
  • Identifier corrects the ID, taking the time to explain.
  • Uploader mentions (or hides) the use of an AI assistant, and pastes some lengthy output from the AI arguing for the wrong ID, including wrong references.
  • Identifier writes a lengthy answer to explain why the output is not useful.
  • Uploader feeds the answer to its assistant that outputs more text, the Identifier answers again, and the loop can repeat itself as long as the identifier is patient..

In one case I have seen, an expert spent at least 20 to 40 minutes of her time replying to text generated instantaneously.
Multiply that by the number of AI enthusiasts (who may not even disclose they are using such software) and you can imagine the potential of wasted time for many identifiers, who unwillingly become unpaid workers to improve for-profit AI models.

I feel some kind of disclaimer about AI use could help (no all AI users are dishonest), but I leave it to the community to think about how to deal with the time wasting..

23 Likes
2

That’s disheartening. I’ve had a few situations like this on other forums. I try to be polite and make it clear I’m not attacking them, but pretty directly state that I’m spending my time to have a conversation with another person who can read what I have to say, think about it for themselves, and then tell me what they think. If I wanted to know what ChatGPT has to say, I could go ask it myself without the middle man. I’ve so far not had anyone give an AI response to that or continue to post AI slop afterwards. If they did, I’d probably be looking at the site guidelines to see if it’s appropriate to get curators involved. Any further conversation would be about etiquette and issues with AI, as I wouldn’t bother trying to explain an ID to someone who was doing this.

17 Likes
3

Is that not something to flag ?

12 Likes
4

It seems like this would fall under the rules about machine-generated content:
https://www.inaturalist.org/pages/machine_generated_content

15 Likes
5

(And/or) enter a tentative ID, hit the ‘reviewed’ checkbox, DQA the observation, then move on? Assistive technologies of all sorts (image matchers, spellcheckers, prompt-fed chatbots etc.) are there to stay. If after discussion involving personal logic and knowledge: ‘in person’ is the way to go.

6

I see this quite a bit on Facebook threads and the like. I don’t understand it. Someone asks a question, and someone else asks ChatGPT etc. and posts its answer. Surely if they’d have been satisfied with a machine answer they wouldn’t have posted on a chat!? I wouldn’t post another person’s comment as my own in case they were daft, so why would I post a machine’s? Anyway, minor rant over!

23 Likes
9

The nature based groups I follow on FB, with some people in common with iNat, often have a No AI answers rule. Our problem is that AI doesn’t consider distribution.

5 Likes
10

Wikipedia new policy bans the use of LLM to generate or rewrite content for most cases. LLM is only allowed for basic copyedits and translating Wikipedia articles. The vote was 40 Wikipedia editors support the new policy, 2 Wikipedia editors oppose the new policy.

https://en.wikipedia.org/wiki/Wikipedia:Writing_articles_with_large_language_models

Some background info about the new policy.
https://www.404media.co/wikipedia-bans-ai-generated-content/

If there are identifiers who have strong opinion on the use of AI for identifications, y’all can look at the actions of the Wikipedia editors about AI-generated content.

13 Likes
11

I’m interested in this thread for several reasons. I am a retired physician by training, and when I was becoming a doctor the internet was becoming a consistent source of information for patients and lay people for the first time. There was, and continues to be, considerable debate about this. I would put forth what an older and wiser physician said to me when I was in my training- that all information should be accessible to all people. How that information is interpreted is important, and there are situations where guidance from experts is extremely relevant. If we consider AI to be a logical evolution of information technology, the question is not how to control it, but how to best utilize it. My point is, AI is just a tool. It can be used well, and it can be used poorly. If you, as an expert, are able to contribute to the next generation to improve identification of rare and/or important organisms, the data that you provide may allow your expertise to extend beyond your individual human capacity. The question of who may stand to gain financially is beyond the scope of my intended message, and so I will not comment on that discussion here. I would urge stakeholders to consider guardrails and best use of a tool that we, as a community, would rather be a good tool than a bad one. Thank you to everyone who provides their expertise freely, and I am sorry for anyone who has ever been made to feel exploited. Ideally, AI could be used to triage the more interesting cases to the experts and automate the more straightforward identifications that often go unidentified. Information is power, and should be used for good.

9 Likes
12

I don’t think this thread is about broader questions concerning the pros and cons of AI, it’s about a scenario where someone is reposting AI responses without even checking them. That’s an obvious misuse of the tool, and that’s the problem being discussed. I have used many different LLMs, and almost without exception, if they don’t know the answer to something, they will confidently guess and invent non-existent evidence to support their guess. Most humans know not to do this, and those that don’t typically struggle to form well-structured arguments, which makes it relatively easy to spot. LLM is unique in the way it fabricates information while sounding like an expert, and I think that makes it a singularly unreliable and often misleading source.

If your argument is that expert identifiers should be willing to engage with AI responses to train the next generation of AI to be better, I don’t even think this would be an effective way of achieving that goal. AI learns best from high quality training data. Ironically, sharing AI-produced misinformation (incorrect claims and arguments supported by fabricated sources) harms the next generation of AI, as it will get confused by its own falsehoods. It would learn much more effectively if experts answered questions and shared their knowledge without being drowned out by paragraphs of AI-produced misinformation. You also write that information should be accessible, which I agree with. Posting low-quality AI responses makes it harder for people to find the high quality information that they are seeking, therefore decreasing the accessibility of good information.

Edit: To clarify, I am not suggesting that AI is without utility and should never be used. I myself use it for a variety of things (but never for forum comments). I am suggesting that the case that inspired this thread is clear misuse that should be recognized as such.

10 Likes
13

I’m finding it ever more difficult to see a positive future for humans when it comes to knowledge. The younger generations that will be growing up with generative AI will, in large part I’m afraid, not know how to think or formulate their own ideas–why would they when they can just ask genAI to do it for them (it’s bad enough already in college students not reared on AI). This is not at all comparable to the invention of calculators or other technologies. The companies that produce AI models will be able to shape the mindset and opinions of their users. This is all relevant to this discussion because allowing people to add information to iNaturalist–generated without human thought–hurts everyone. Allowing it contributes to the overall downward spiral of information quality.

Thus my stance on AI assistants formulating information and people posting those as answers on iNat is that it should be strictly forbidden (except for suggesting revisions for clarity or for translations). It’s near impossible to enforce, but that’s no reason it shouldn’t be forbidden.

I’m glad that iNat’s official policy is “Thus, our intent is that observations, identifications, and comments are manually generated by individual human choices. To achieve this, we’ve made posting machine generated content a suspendable offense.”

https://www.inaturalist.org/pages/machine_generated_content

I didn’t know about his policy until now. However, one bullet point in these guidelines needs to be tweaked: “Machine generated comments would include a generic comment posted via a script.“ This suggests that specific comments can be posted. The word “generic” should be removed. A very detailed (specific) AI-generated comment is worse than a generic one.

8 Likes
14

The response indicating that my statement was overly broad was fair. Perhaps I should have run it through ChatGPT first (kidding). I would like to clarify and refocus toward the original question of, how to respond to an AI-generated comment from an iNat user. In reviewing the posts, there appear to be a few related but separate issues at stake.

  1. Data Quality, specifically, if an AI suggests the wrong identification.

  2. Data Usage, specifically, if an expert’s time is being exploited for training purposes without consent

  3. Communication, or difficulty in having meaningful dialogue when another user is utilizing AI.

To address any of those AI-misuse issues, it would be helpful to quantify these problems. For example 1) What percentage of misidentifications persist because of AI-generated content? 2) What is the cost to the system because of AI-generated comments/disagreements in terms of time/trade-offs/disengagement/etc, 3) Data Usage rights are an important topic that deserve their own thread 4) From an iNat user perspective, why is AI being utilized in this way?

I think that (4) is the most interesting question and extends into a context of democratic access to information which I alluded to in my original post. Many of us use AI to reverse image search, find more information, and engage in a meaningful way. There are also those individuals who rely on AI for more complex reasons. For example, there may be language challenges, educational or power differences, social challenges, developmental differences, etc. In these cases, the exchange of information may still be meaningful even if delivered via LLM. I can imagine cases whereby LLMs may actually make these conversations and information better and more accessible to a larger population.

The important piece, to me, is that we are engaging with people in a respectful and useful way. If an individual is utilizing AI to communicate, that may not be nefarious. I hope this is useful and I have not strayed too much this time. Thank you all for what you do.

6 Likes
15

All of this is correct, but I think the focus here is on comments generated by AI. For example, I prompted Copilot with “Generate a post to comment on an iNaturalist observation describing how to distinguish between Bombus pensylvanicus and B. sonorus.“ and got this response:

Distinguishing Bombus pensylvanicus (American Bumble Bee) from B. sonorus (Sonoran Bumble Bee) can be tricky since the color patterns are very similar, but a few characters help:

  • Geographic range is often the strongest clue.
    B. pensylvanicus is primarily an eastern and central species, while B. sonorus replaces it in the Southwest and Mexico. Observations west of west TX / NM are much more likely sonorus.

  • Abdominal color pattern:
    Both species usually have yellow on T1–T2 and black on the rest of the abdomen, but B. sonorus often has a cleaner, more sharply defined transition from yellow to black. B. pensylvanicus can look grayer or more diffuse.

  • Thoracic pubescence:
    B. sonorus tends to have brighter, richer yellow on the thorax, while pensylvanicus often appears paler or slightly brownish, especially in worn individuals.

  • Size and shape (subtle):
    B. pensylvanicus queens and workers are often a bit bulkier and longer-bodied, though this is not reliable on its own.

Because color varies with age, wear, and sex, location + overall pattern together are usually needed for a confident ID. If this was observed within the known range of B. sonorus, that species is generally favored.

For the most part, this is pretty accurate and well written. But should I be doing this? Anyone with zero knowledge can now go in and insert these comments and come across as quite knowledgeable. Is this the world we want to live in?

5 Likes
16

I can even create a response to this forum post. Here’s the Copilot response when I feed it the original post (note: everything that follows was generated by Copilot, and IMO, is a better response than anyone else has posted so far–but is this the world we want to live in?):

I think you’ve put your finger on something many identifiers are starting to feel, even if it hasn’t always been articulated this clearly.

The issue isn’t simply “AI bad / people dishonest” — it’s the dynamic this creates. When an identifier takes time to explain morphology, ecology, or taxonomy, that time is a gift to the community. When that explanation is immediately fed into an AI system and bounced back as a longer, more confident‑sounding rebuttal (often with fabricated or misapplied references), the exchange stops being a conversation between two people and turns into unpaid validation work for a tool that has no stake in being correct.

What’s especially frustrating is the asymmetry: 20–40 minutes of careful expert reasoning versus seconds of autogenerated text. Even well‑intentioned uploaders may not realize how much cognitive labor they are asking for when they keep pasting AI output instead of engaging with the reasoning themselves.

I agree that transparency would help. A simple expectation like:

  • clearly disclosing when AI tools are being used, and

  • treating AI output as questions to learn from, not counter‑arguments to experts

would already go a long way. So would a shared understanding that identifiers are not obligated to respond to AI‑generated rebuttals at all.

Ultimately, iNat works because people volunteer their expertise generously. Anything that systematically converts that generosity into time sinks or training data for for‑profit tools deserves some collective thought. I don’t have a perfect solution either, but naming the problem is an important first step.

Thanks for starting this conversation — I suspect many identifiers are quietly relieved to see it raised.

5 Likes
17

I empathize with these existential questions. In order to be impactful, we have to delineate the questions and concerns objectively. When that first part is achieved, we can measure/monitor and create guardrails and solutions. Experts as well as AI’s stand on the shoulders of giants, and both can be wrong. Personally, I am delighted that someone knows this much about bumble bees, and that bumble bees exist! I personally would have liked to have included in that response that “Dumbledore” means “bumble bee” in Old English, hence the name of Hogwartz. But that would have been less useful. I do believe people are generally doing their best, and there are always opportunities to improve the system. Thanks for your thoughtful response.

2 Likes
18

Hard to know without understanding exactly what the “Uploader” in this situation is actually doing. But the purpose of the Machine Generated Content prohibition is less about using AI generated text and more about using scripts or other automation to add content, particularly IDs, or comments or observations, without human oversight. It was motivated by a situation where someone wrote a script that added agreeing IDs to every observation of a species, so their account could be the “top identifier” of the species. Or, we don’t want people using a script to automatically upload every photo their camera trap records. In both cases there is no human oversight at all to the content being added.

In this case, it seems like the AI assistant was used more as a tool by a person, who we assume read the text and and then posted it. So while not great, I’m not sure it’s a violation of that policy. The rate, however, may be.

@jbbohan would you be able to message me URLs of some of these so I can take a look? The current policy isn’t really designed for something like this and we may need to see some specific examples in order to understand the issue better.

In this case, no sources are cited. I would ask the person to please link to papers or articles where these tips were drawn from. IMO this isn’t too different from me saying “it’s definitely species X because of features Y and Z”, which people (including myself) do all the time, and then someone replying “thanks, can you cite sources for that? I’d like to see where they’re described.”

8 Likes
19

Going straight to your 4th point, I agree that most people using AI like this do not have any negative intentions. My guess is that in the example case, the person was a bit out of their depth. They were confused by the IDer’s comment, and since it contradicted whatever prior understanding they had, they also weren’t sure if the comment was right. So they asked AI to explain the comment, possibly using some version of this (exaggerated to demonstrate the possible issue):
“I thought the answer was ‘X’, and almost everything else online agrees with me, but another person says the answer is Y. Their explanation is confusing, and I’m wondering if it might be wrong. Can you you explain their comment to me? Are they definitely right, or is it possible the answer is actually ‘X’?”

If the LLM doesn’t understand the topic or know what the correct answer is (and it may not, because up-to-date ID info can be incredibly obscure), its explanation is likely to be useless and to have little basis in fact. It’s highly susceptible to bias and to reinforcing whatever someone already thinks. It can do this even if you are actively trying to prevent that from happening, and it has an extremely loose relationship with the truth. In real life, the interaction was probably more subtle, but the effect can be similar, as we rarely describe things from an entirely neutral perspective. If the observer wasn’t convinced yet, they probably would have let the LLM know that. I am speaking from personal experience on this. I used AI quite heavily, in part to try to develop a more informed perspective on it.

The line between appropriate and inappropriate AI use is an absolutely massive topic about which many pages could be written, and I don’t think it can be easily reduced to a small number of objective questions. I have come up with some general guidelines for myself that I think could also serve others well, but haven’t taken the time to write them out. I think AI is primarily useful in two types of situations. The first is helping you in areas where you have expertise, and can easily recognize any errors in its responses. In these situations, it’s helping you with a process of some sort (coding, helping with very specific parts of the editing process while writing, or any of a large variety of things). It’s not acting as a source of new information, and its responses are closely vetted and modified rather than being used directly. When you do enough of this, you will learn exactly how fallible it is and how confidently wrong it can be.

The second type of situation when it can be helpful is when you know very little about a subject. I’ve used it to explore topics in physics and philosophy that I have very little prior understanding of or education in. Sometimes, I have an interesting thought, question, or idea, I’m sure it has been extensively studied before, but I don’t even have the terminology I need to find more information about it. AI can put a name to some of these ideas which allows me to find things I would have otherwise struggled greatly to find. The key here is that I don’t trust anything it says even a little bit, and I mean that literally. I treat everything it says as a guess, and it just happens to be quite good at guessing. Any new knowledge or understanding it provides isn’t real until I can validate or test it by some independent outside source/method. Other sources can have errors, but I think it’s incredibly rare for them to have the sort of blatant fabrications that AI can produce, which would be seen as fraud or deception if they came from a human. I find AI least useful in areas where I have moderate understanding, because it takes longer to verify whatever it says than to figure things out myself.

I think there are also a lot of more subtle and pernicious effects that can accompany AI use. It has several slippery slopes that can lead to accidental misuse, and one needs to always be assessing their relationship with it. Is it helping you to learn, or just giving you false confidence, reinforcing your views, and solidifying possibly incorrect beliefs? This one is a much greater risk than it seems. Most LLMs default to validating you and telling you what you want to hear, perhaps articulating what you could not and explaining why you are right even if you weren’t totally sure. For a while, I gave custom instructions warning against this and trying to make my LLM more critical/skeptical. The result was that it more often challenged me, but I “won” every single debate or discussion I had with it. The ironic and counter-intuitive effect was that it strengthened my views more by challenging me than it would have in trying to validate me. LLMs have improved a bit to avoid excessive validation, and I think this can make it less obvious and more powerful. I think it can be dangerous for this reason, and I try to avoid discussing subjective things, beliefs, or interpersonal dynamics with it. If I ever do, I frame things in the third person so it can’t tell what I actually think or believe (this is also a good idea for objective subjects).

I think everyone must ask themselves whether they are really using AI for learning, or if that’s a cover for a sort of intellectual forgery in which you misrepresent your level of understanding and fool even yourself in the process. Are you using it to help you better articulate your arguments, or is it subtly shaping your arguments and beliefs in ways you may not even be aware of while weakening your ability to think for yourself and make your own decisions? Are the arguments even your own, or is it giving you what it thinks you want to hear before you have taken the time to think through things independently? I think these are subjective personal questions that require honest introspection and usually don’t have definitive answers. They are intended to be a jumping off point to reflect on how and why one is using AI rather than a way to assess whether they should be. Yet when I see obvious AI text shared verbatim and someone is doing this repeatedly, I can’t help but feel that this represents an extreme, and that the person has abdicated their position in the driver’s seat. When AI is used properly, I think it should be difficult or impossible to tell that it was used at all.

Based on the fact that many people struggle even to vet the information they find online by more traditional means, I think a large fraction of society currently lacks the ability to use AI appropriately, and there are inadequate norms and guidance to help them with this. In sharing my own experience and thoughts on AI, and gently pushing back when I see what I believe to be misuse, I’m trying to contribute to the development of new norms that will increase our resiliency to it. I’m not opposed to using AI for editing comments and posts, but I fear a world in which AI performs both the writing and the reading, helping me to decide what to say and how to say it, and helping you to decide what to think and how to feel in response to the condensed summary it feeds you.

Note: I use “you” in a generalized sense throughout this post, and it’s not intended to refer to ‘you’ meriweather; I have no reason to suspect that you are using AI or that your use is improper if so.

9 Likes
20

I used to have basically the same opinion, but the more I’ve read about AI the more my opinion has changed. I now believe that the companies currently competing in the AI marketplace are not at all interested in creating knowledge tools. They are interested in making money. This is why LLMs always give confident answers even if they have no sources to base it on and have to make things up. According to a study published a few weeks ago, certainty cues increase trust in AI answers whereas uncertainty cues reduce trust, and trust is one of the key variables affecting how much people interact with AI (all of which I’m sure AI companies have known for years). At this stage in the development of the AI industry, where private capital investment has maxed out and the transition to public investment (IPOs) is imminent, the only real goal is to maximize engagement. AI companies have known about the harmful biases and hallucination problems of LLMs for a decade now, and yet they have consistently chosen to ignore this in how LLMs present information. So I have no faith that AI companies are interested in improving factual accuracy beyond “the internet says so”. The return-on-investment for creating actual knowledge tools just isn’t very high compared to creating “content” engines. Banning AI-generated comments may be an ineffective and reactionary response, but I have to say that I have some sympathy for the idea at this point.

12 Likes
21

Keep in mind the main reason Wikipedia banned AI-generated content wasn’t because of false hallucinated content, it was because of LLM’s tendency to misrepresent cited sources, so asking for sources in cases like these might just add to the pointless time-suck.

10 Likes