The new Google Gemini seems a little dumb in some ways compared to ChatGPT, but interesting to see the option to upload images without subscription. I wondered if the species IDs might be somehow pulling from iNat, so I compared the two.
With the integration of Gemini into gmail and docs… and the new meta.ai into WhatsApp and Instagram…I guess automated species ID for the general public might happen more commonly within the social media apps we use before long ( which would be a shame potentially, as the data may not then be preserved ).
I imagine many of the engines will ultimately be used from a single application that does what you just did. Looks at the results from several of them in parallel to id a species. In the same way as field guides from different publishers or keys from different botanical societies have always been used.
Which Id makes the most sense to the human at the center of it all, and why?
I actually have been doing some testing with GPT-4 Vision for a personal project on the iNaturalist Open Dataset. It managed to ID only about 28% of the 1000 random images I tested it on correctly, which is pretty decent for a “generalist” tool. I have some doubts that any of these generalist vision models are going to be able to compete with a specialized tool like iNat’s CV any time soon (or maybe ever) but who knows, nobody can predict the future.
In the near future, I think you’re probably right. That said, all it will take in the future is for one of the large general models to “take an interest” in nature photos, perhaps “of it’s own volition”. The iNat amazon open dataset is out there for any model to be trained on, and good training data is really the biggest limiting factor in model development. So in a few years who knows?
I tested Gemini with some easily identifiable spider photos and told it the location of each one. Its responses were utter nonsense and none of its ID suggestions were even in the right genus (and most were implausible given the locations). It was able to ID to family correctly, but that’s relatively trivial. Some of the information it provided was laughably wrong, for example, claiming that Black Widow Spiders (Latrodectus mactans) and Brown Widow Spiders (Latrodectus geometricus) are jumping spiders. It was also interesting that virtually all of the species it suggested were from Europe or the United States, even when I told it the spider was from elsewhere. I would say its accuracy is about as good as asking an 8-year-old (with a European/American bias).
The only thing I’ve found Gemini to be semi-useful for is coding suggestions, and even then, it is still inferior to Chat-GPT. I see the “AI” responses to Google searches (which are from Gemini) for nature-related searches I do, and they often contain errors. I am not surprised Gemini is awful at IDing pics, but it’s laughably bad that you even told it where the spider was from!
I’ve noticed that Apple’s Photo application has improved at ID’ing moths in the past few months. A year ago it almost always ID’ed a moth as a European genus/species. Now it frequently gets the species right.
It is probably only a matter of time until someone gets poisoned/envenomated by an AI incorrectly IDing something. I am surprised we haven’t heard of it yet, though there are papers warning about it: https://qz.com/ai-mushroom-id-dangerous-1851364485
Well that is truly horrifying and a good example of why I tell people to never ever trust AI id tools for mushrooms without independently verifying the suggestions.
What’s even more horrifying is he didn’t do the slightest checking as death caps and puffballs don’t exactly look alike. I’ve tried to use co pilot, chatgpt, and google ai to help with identification but generally find it completely unusable as it often just makes up information including the scientific names. It really scares me the hype behind AI with people not really understanding the tech or taking the time to fact check what it spits out.
beyond the immediate risks of spreading misinformation, disincentivizing critical thought, and, yes, encouraging people to eat lethal mushrooms – there’s also the huge problem of how megacorporations’ insatiable lust for profits WILL influence the results. Google is already known to change its search results based on the political affiliation, age, sex, race, etc of each user.
iNat is imperfect, but to my knowledge it isn’t harvesting my data to sell to the highest bidder. Nor does it spend hundreds of millions of dollars buying politicians. I think I’ll skip Gemini.
AI isn’t evil, it’s a tool – but every tool built by humans has human biases by default. iNat CV isn’t built for profit, and it’s better for it.
That’s the thing though, sect. phalloides amanita are considered look alikes for puffballs before they emerge from their volva. Basically every resource instructs foragers to slice all small puffballs in half to make sure that there aren’t any developing gills visible in a cross-section.
And I could definitely see an AI mistaking a white amanita for something like Leucoagaricus leucothites (which beginners shouldn’t be eating ANYWAY because of how difficult they are to distinguish from destroying angels) or Volvoplueteus gliocephalus (which has very pale gills before spore maturation.)
People using AI for mushroom IDs gives me a major eye twitch
I suppose eye twitch is better than a fried liver.
I don’t really use iNat for fungus IDs, but has anyone noticed if iNat performs any better than other generic AIs specifically when IDing fungi? (not that I would recommend relying on any AI for fungal ID). It would be interesting to note if there’s at least a difference in anecdotal performance. I expect iNat would at least have a better training library than Gemini/ChatGPT for fungi IDs.
On a side note, I did find it interesting that in the article @choess posted, it said “He pulled out his smartphone, photographed the fungi and uploaded the image to a plant identifier.” That might be the first problem…
