Does this exist? If not, who’s working on it?
I am thinking of a phone app that looks at your geographical location, and suggests species ID’s for cicadas or orthopterans based on the courtship sound of the male.
Can iNat’s “computer vision” do this? Would it then be “computer hearing”?
I know that bats have an app that does this, for feeding (echolocation) sounds
what is the app for Bats?
https://www.aquila-ecologie.nl/birdnet-voor-sprinkhanen Om het leren van deze geluiden veel toegankelijker te maken, heeft Aquila Ecologie, in samenwerking met Provincie Limburg, zich toegelegd op het maken van een Android app die met AI de geluiden onderscheidt. De 35 soorten die de app kan herkennen worden met 90% nauwkeurigheid onderscheiden, wat uitzonderlijk hoog is. De app zal, wanneer deze beschikbaar komt, bij een herkenning foto’s van de soort laten zien en een tekst met basisinfo.
Met de nauwkeurige herkenning van geluiden wordt opeens ook mogelijk om lange opnamen te analyseren en daarin alle soorten te herkennen. Daarmee kunnen allerlei onderzoeken gedaan worden naar het voorkomen en activiteit van sprinkhanen die eerder niet mogelijk waren.
Dat de herkenning zo nauwkeurig is, komt door een speciaal voor dit doel ontwikkeld algoritme van Aquila Ecologie. Waar vogels herkend worden op basis van spectrogrammen (visualisering van de toonhoogtes over de tijd heen), werkt het algoritme van Aquila Ecologie op basis van de ruwe geluidsdata. Het ontwikkelen van zo’n algoritme is veel werk: er zijn oneindig veel verschillende mogelijkheden van verschillende laagjes van bewerkingen die uiteindelijk leiden tot een herkenning. Maar het is ook de uitdaging om de juiste laagjes op elkaar te stapelen om zo de best mogelijke resultaten te krijgen.
De app voor Android zal waarschijnlijk rond juni 2024 beschikbaar komen.
https://www.aquila-ecologie.nl/mogelijkheden/sprinkhanen/
Blockquote Om de kwaliteit van onderzoek naar deze soortgroep te versterken, heeft Aquila Ecologie software ontwikkeld om de soorten op basis van geluid te onderscheiden. De herkenning heeft een zeer hoge nauwkeurigheid, doordat we een eigen, krachtig algoritme gebruiken. Alle voorspellingen kunnen naderhand handmatig gecontroleerd worden.
Blockquote
Aquila Ecologie kan een geluidsopname, bijvoorbeeld verzameld tijdens een SNL-insectenmonitoring, automatisch analyseren. Aquila Ecologie heeft ook de expertise in huis om de geluidsopnamen te controleren op onjuiste determinaties. Tegen lage kosten kan daarmee een nauwkeurige bepaling van het voorkomen van sprinkhanen en krekels gemaakt worden.
https://www.aquila-ecologie.nl/mogelijkheden/sprinkhanen/
And fish sounds HYDROFOONS (Canada, I forgot)
Na meer dan honderd uur geluidsopnamen en een literatuurverkenning zijn we in staat om met de hydrofoon een aantal
soorten te herkennen: baars, snoekbaars, meerval en brasem. We
horen ook andere geluiden, maar weten nog niet wat we dan horen.
We weten ook dat het dankzij menselijke activiteit erg lawaaierig
kan zijn o Door visgeluiden uit Hydrofonos te koppelen aan de vangst wil de
RUG onderzoeken welke geluiden bij welke vissen horen. Dat is een spannende en
gecompliceerde zoektocht. In de Waddenloods laten we enkele geluidsfragmenten horen. https://natuurtijdschriften.nl/pub/656377/SchubbenEnSlijm2017009001007.pdf
I’m not the most computer-savvy person, but in case you don’t know how to translate a post into English, I think Andre Hospers (@ahospers ) wrote:
Https://www.aquila-ecologie.nl/birdnet-voor-sprinkhanen
To make learning these sounds more accessible, Aquila Ecology, in collaboration with the Province of Limburg (Netherlands), has focused on creating an Android app that distinguishes these sounds with AI.
The 35 species that the app can recognize are distinguished with 90% accuracy, which is exceptionally high. When it becomes available, the app will show photos of the species, as well as a description with basic info.
With the precise recognition of sounds, it is suddenly also possible to analyze long recordings and recognize every species present. This allows all kinds of studies to be done about the occurrence and activity of orthopterans that were not possible before.
The fact that the recognition is so accurate is due to an algorithm specially developed for this purpose by Aquila Ecology. Whereas birds are recognized based on spectrograms (visualization of the pitches over time), the algorithm of Aquila Ecology works based on the raw sound data.
Developing such an algorithm is a lot of work: there are infinite possibilities that lead to recognition. The technical challenge is to stack the right sound layers together to get the best possible results.
The app for Android is likely to be available around June 2024.
@merowig The app for bats is called Echo Meter Touch 2.
You can watch a 2-minute Youtube video about it here:
I would doubt it. Considering that at this time, Merlin is notoriously inaccurate; just look at the mess that eBird is now from people uploading their unconfirmed MerlinID birds demonstrates that.
So I think we’re still quite a ways off before we have something that can distinguish insect sounds. Even with images AI still struggles: I’ve been looking into BeeMachine, and it still has a rather high rate of inaccuracy even though the area I’ve been examining only has 11 species.
thank you!
This might be difficult for a variety of reasons.
Many things have to happen first before this is possible, even for more recognizable groups. Audio recognition often uses similar models as image recognition, where the computer is trained on spectrogram images of the audio rather than “hearing” the audio. So the first step would be to have iNat generate a standardized spectrogram (proposed here; discussion of logistical hurtles can be read about in that thread). Then some form of moving-window recognizer has to be made because it doesn’t really work to have the computer look at the entire audio clip (it needs to take it segment by overlapping segment).
Honestly, I have never thought about this, but considering how useful Merlin is (although it has its flaws) I think this would be a great app to have.
I agree that a machine learning model to do this task would be very useful but quite challenging to develop (well) due to the really high diversity of insects and the sounds they produce.
An app based on such a model might be very cool, as long as users don’t upload screenshots of the app’s suggested IDs as observations on iNat 
I approve your translation, but I thought this app already exists and could be installed on an android phone:
https://play.google.com/store/apps/details?id=com.aquilaecologie.sprinkenapp1
I found a review for this app in a thread
https://forum.waarneming.nl/index.php/topic,507173.0.html and I think this app needs some improvements.
And also the work on create a model for bat sounds, but that will take a year or two…
https://www.aquila-ecologie.nl/projecten-2/
Is Merlin inaccurate all over the world or just outside Europe and the USA?
(Neylon seems to be based in the USA…)
Useful website https://orthsoc.org/sina/
If memory serves, the developers for MerlinID consider Merlin “trained” on a bird when they have 100 clean recordings to train it on. There are quite a few North American species which they don’t 100 clean recordings of, several species of warblers for example. I would imagine that the rest of the world likely has even fewer “trained” species.
Compounding the issue (as I understand it), is Merlin gets its training for what species to expect in an area from eBird; where people are uploading whatever Merlin heard without verifying. That is a rather scary cycle when you think about it.
Now, the issue people usually talk about regarding Merlin’s questionable ID’s is the rare birds: we have a guy in my county that is driving all of us nuts with the worthless rare bird emails. However, I’m not as concerned about those, the eBird reviewers are cleaning those out. The ones that scare me are the spike in Carolina Wren, Northern Mockingbird, Eastern Meadowlark, and Bobolink records we’re getting. Those are all birds that we expect in this area, but not in the numbers being reported or in the places (Bobolink and Meadowlark in forests? right.) So we’ve got birds whose ranges are expanding (Carolina Wren and Northern Mockingbird), or shrinking (Bobolink), but we are now unable to use eBird as a resource to track this, because of a massive amount of people mis-using Merlin.
So I tell people to think of the Merlin app as if you are birding with a pretty good birder: If we were out and I said “I think I heard a Golden-winged Warbler”, you wouldn’t just check it and move on, you’d listen for it to and maybe try to see it. It’s a useful tool, but if you didn’t verify it, it wasn’t there. I think it was Reagan that said “trust, but verify”, for Merlin I’d change that to “doubt, but verify”.
Sound ID will be expanded in the future to include species worldwide, but to do that, our team needs a minimum of 150 sound recordings for each species to train Merlin to recognize their sounds.
Last three months I see many people using this app so I think that after three years the recognition is so good that people install the app ‘Merlin Bird ID’ and keep it.
Let’s not turn this into a thread about Merlin, please, unless it’s related to the original question.
I agree that insects are probably going to be difficult for the reasons stated above. Something for other vertebrates like frogs and mammals might be more feasible, although I’m certainly no expert in this area.
Frogs calls are definitely ripe for machine learning models to identify calls. Multiple models have been created that do this, though none are worldwide that I am aware of.
This dataset for Australia is probably the biggest:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7040047/
Here’s one from Japan:
https://meridian.allenpress.com/copeia/article-abstract/111/4/563/496845/Evaluation-of-Deep-Learning-Based-Monitoring-of?redirectedFrom=fulltext
There are some other “bespoke” ones I’ve heard of that are made to ID a specific species of interest from frog loggers. So I think frog call models are certainly possible with good training sets.
https://openresearchsoftware.metajnl.com/articles/10.5334/jors.154/
https://www.youtube.com/watch?v=z5C3wLsyGdE&t=50m
https://photos.app.goo.gl/iCaBSZHkzw9cNW9K9
And fish souds (I forgot)
Bij het visonderzoek wordt naast de traditionele manier van vis vangen met schietfuiken en
kubben gebruik gemaakt van hydrofoons. Door visgeluiden te koppelen aan de vangst wil de
RUG onderzoeken welke geluiden bij welke vissen horen. Dat is een spannende en
gecompliceerde zoektocht. In de Waddenloods laten we enkele geluidsfragmenten horen. https://natuurtijdschriften.nl/pub/656377/SchubbenEnSlijm2017009001007.pdf
https://swimway.nl/
https://www.bnnvara.nl/vroegevogels/artikelen/ook-vissen-kunnen-zingen
Do people do that with Merlin? I just upload the audio itself, that way if it isn’t what the app thought it was, I can still get an ID of the bird it actually was.