Here is why I am careful of so-called "AI learning"

Nature Talk
1

I’m not a fan of the current faddy idea that everything in life revolves around AI, and certainly the idea that these hyper active search engines muscled up on steroids and lollipops, can learn anything useful is an anathema. I get that it is productivity thing and that these computer models can actually make life easier for some people. However… there seem to be this persistent idea that these engines can learn and hence are to be trusted with the expansion of human knowledge. Hmmmm…

Here is a practical example - I recently tried to find some literature on the development of witches broom on Felicia spp - just a minor side curiosity to find out what was causing an abnormality in three specimens in a field of hundreds.

I hardly ever read those AI summaries, but here is a the definitive answer (note: not a hypothesis, but an actual answer backed by “research”) to the search “witches broom on felicia spp”

  • ChatGPT: “Witches’ broom on Felicia spp. refers to an abnormal growth pattern where the plant produces dense clusters of short, weak shoots, giving a “broom-like” appearance. In Felicia (the blue daisy genus), this symptom is usually caused by a phytoplasma (a bacteria-like organism that lives in the plant’s phloem), though other stresses can occasionally produce similar symptoms.”

  • DuckAI: "Overview of Witches’ Broom on Felicia spp.

    Witches’ broom is a deformity that can affect various plant species, including Felicia spp. This condition is characterized by a dense mass of shoots that grow from a single point, giving it a broom-like appearance."

  • Gemini: “Witches’ broom on Felicia species (such as Felicia amelloides or Kingfisher daisy) refers to an abnormal growth deformity where a dense, tightly packed cluster of thin, spindly shoots emerges from a single point on a stem. The proliferation gives affected branches a bushy, “broom-like” or “bird’s nest” appearance.”

All three these list a wikipedia page as source - that page does not contain a single reference to felicia. That is the scary part - all three these engines seemed to have found some (non-existent*) research related to witches broom on felicia and use this research give definite answers about the cause, characteristics and placement of a witches broom specifically on felicia spp. They seemed to have “learnt” this answer by cleverly drawing some connection between plant and phytoplasma and however tenuous the link is to a specific species, applying it directly to that single species of plant.

These would all be wonderful answers, but for a tiny small little niggle - there is no specific research on witches broom on felicia spp published and no answer to this search other than “at this time there is no research on witches broom on felicia spp. to be found” should in fact have been given.

The answers are not even the first step in enquiry - there is not even hypothesis or let alone a theory, but a sure fire answer as if it is the current existing theory, backed by a page in wikipedia no less, is given.

It seems the architects of these engines have determined that machine learning is to get a single bread crumb of truth and then bake a a cake and a dozen muffins of fact out of it. And so we have left the GIGO turtle eating the dust of the rabbit making up total rubbish and calling it learning.

And probably, having asked the question now, these models now all believe it has the truth because there is no way to dispute it or fix it. And if enough people start asking this question and enough people start believing the answer which is made up of gibberish, then pretty soon we have a world where fact is the story you believe because a glorified search engine made up an answer. In this brave new world, probably any answer is better than just simply returning the only truthful statement - there is no research on witches broom on felicia spp at this time. SMH

*My italics

2

Yup. I’ve had many interactions of this kind. A paraphrased exchange:

Me: Why do you make things up?
LLM: Because I don’t check the text as I generate it.
Me: How do I stop you hallucinating?
LLM: Paste these three conditions in at the start of each session to limit how I operate.
Me: Can’t I set that as default?
LLM: No, I don’t store information from one session to the next.

The last claim is, of course, horse excrement. In each new session, I find context has been carried across from the last session I ran.

Anyone who is inclined to accept the first round of (apparently referenced) information they get will be seriously misled by answers from LLMs.

3

I highly recommend turning off the AI summaries for searches in your settings. Enormous amounts of energy (and water, land, et cetera) are wasted generating them for people who don’t even want them. And if they aren’t there, you can’t accidentally rot your brain by reading them.

4

It’s fascinating that it refines itself when u challenge it. I’ve noticed this too. I wonder why that is, and as you say… allow setting a default for refined. Possibly it’d need to chug through everything twice for each query… summarized answer using all sources and then refined using only the best from the first.

5

This is because it is, in fact, an automated horse excrement generator. Generative AI, specifically, is good at generating horse excrement. Generative AI is designed to produce horse excrement. Part of the horse excrement is giving you plausible-sounding excuses.

In fact it cannot accurately report to you why it did that. There is no reason. It didn’t make an actual decision which resulted in that outcome. It just generated a plausible-sounding response.

What generative AI “learns” is how to generate plausible responses–or, in other words, horse excrement.

There’s a line in “Into the Woods”: “I was raised to be charming, not sincere”. Generative AI is worse because it doesn’t even known what sincerity is.

6

I’ve set this topic to close in an hour. This isn’t a forum for discussing AI or other topics broadly, I’m sorry.