Merlin for insects?

Nature Talk
22

Yes, people do upload screenshots only.

Uploading the actual audio is definitely a best practice, so keep that up!

23

There is a secret device that records bat sounds, identify them with Tadarida ( or an alternative ) and sent the information. https://x.com/marcelhospers/status/1718401095423127659/photo/1

24

I am unsure if an app like this exists but I do know for plants. This sounds too advanced for the current ML progress but research in bioacoustics and machine learning is ongoing, and institutions like Cornell Lab of Ornithology and other bioacoustic researchers might be working towards similar technologies.

25

The AMBER project, funded by the Abrdn Charitable Foundation, will develop and trial cutting edge technology for monitoring biodiversity. The project will combine expertise from UKCEH and the Alan Turing institute to build systems capable of recognising nocturnal insects, birds and bats using the latest AI methods. https://www.turing.ac.uk/research/research-projects/amber

https://x.com/mclduk/status/1700124444654518377

26

I think the problem is it’s users, not the Merlin app itself. Is Merkin perfect? No! Can I recognize when it identifies a call inaccurately? Many times yes, but then I’m not expecting 100%, just some help and suggestions. just like INaturalist !! Now, I know people who just want an answer, and a wrong answer is good enough for them…. Can’t help such folks eh?

27

https://x.com/mclduk/status/1700124444654518377 Announcing fully-funded PhD positions on our new “Bioacoustic AI” project: http://bioacousticai.eu Apply now for a #PhD studying animal sounds (#bioacoustics), #deeplearning, #acoustic signals, and #ecology! 2 open to apply now, 8 more coming soon. (Pls share)

It seems Dutch. The group, girl who wanted to make an open source catalogue, database of sounds gets her PHD degree and as far as i know she did not succeed to create an open source database. The oterones i know (UK, French) turned out to be a bit closed source fixed to the project.

https://www.sciencedirect.com/science/article/pii/S157495412300287X?via%3Dihub

28

https://bioacousticai.eu/update/the-kick-off-meeting-was-held-in-october-2023-in-leiden-the-netherlands/ This project has received funding from the European Union’s Marie Skłodowska-Curie Action under grant agreement No 101116715

Webdesign by Liesbeth Smit / The Online Scientist

Illustrations Tim Juffermans | studio071.nl

About BioAcoustic AI

  • Marie Skłodowska-Curie Doctoral Network

  • Short name: BioacAI

  • 9 universities/research institutes, 2 museums, 5 SMEs, 3 NGOs, and 1 governmental organisation

  • Duration: 2023 – 2027

  • 10 PhD projects

  • 23 experienced investigators

The blog is interesting

Blockquote As we launch our new “Bioacoustic AI” research project, I thought it a good idea to look over the state of the art in machine learning methods for animal sound. This short article aims to give you an update on interesting machine learning developments I have seen.

Our project is wider than just “machine learning” – we cover electronic devices, animal behaviour, and ecology too – but I’ll focus here on the ML.

Most of what I want to say about deep learning for bioacoustics is contained in the overview I wrote in 2022. I don’t need to repeat all of that – I suggest you read that paper for a more complete treatment, especially if you’re new to this area. This blog article is an update on recent work in machine learning for animal sounds.

Deep learning is getting easier: Off-the-shelf CNNs

I wrote in 2022 that it was becoming common to use an “off-the-shelf” CNN (convolutional neural network) rather than designing your own, and this trend certainly continues. Increasingly this year I find that many projects are choosing the “EfficientNet” CNN – it seems to be a good robust and efficient design. ResNet and MobileNet are also popular.

I also wrote about the increasing use of pre-trained networks. This also continues to be very widespread, and it’s a good thing. One particular benefit of pre-trained networks is that they can dramatically shorten the training time needed for a given task, which may be an important factor in the carbon footprint of this type of research.

Pre-trained networks continue be very widespread, and it’s a good thing. They can dramatically shorten the training time needed for a given task.

BirdNet is a popular birdsong classification CNN model, and various authors have explored its use (directly for classification), and also its use as a pretrained network for other tasks. In the first category, I recommend the 2023 paper “BirdNet: applications, performance, pitfalls and future opportunities”, which looks at the algorithm from a user’s perspective and across various studies that have used it. In the second category, the paper by my colleague Burooj Ghani about “Global birdsong embeddings” looks at big pretrained models, especially bird classifiers, reused for other tasks such as non-bird species. They argue that models pretrained with diverse birdsong are particularly powerful – for them, both BirdNet and Perch are consistent good performers.

These pre-trained networks are often used to provide “embeddings” – meaning, to transform an input into some kind of representation that we can then use as if they were any other kind of numerical “features”. Embeddings are everywhere. I’ll come back to that topic.

Another aspect of “off-the-shelf” is making algorithms easier to use. Animal-Spot (paper/code) is one example of a framework that aims to make it easy to train a bioacoustic ML system hassle-free. It uses a standard (ResNet-18) CNN and provides a pre-designed training setup when provided with a set of audio files.

CNNs versus Transformers: Which to use?

There is no final “consensus” about whether Transformers will take over from CNNs as the best default deep learning architecture for audio. In both cases, the spectrogram is still the most common input data, even though “raw” waveform approaches are possible (and interesting!) for both. However, it’s increasingly common to see good results coming from spectrogram-based Transformers. (What about spectrogram-free transformers? It could happen…)

Few-shot learning shows us how to use many datasets

We have been running the few-shot bioacoustics challenge for a few years now, and it has given us a lot of insights and ideas. I recommend our 2023 journal article about few-shot learning which gives the most complete discussion and analysis.

Some lessons we have learnt from this include:

  • Few-shot learning and “meta-learning” are good directions to investigate because they are one more step towards generalised multi-dataset learning. We now live in a world of many diverse open datasets. Tthe above-mentioned “pretrained” paradigm is a way to make use of 2 different datasets (typically one big, one small). The few-shot paradigm is a way to make use of lots of small datasets. The next step is to learn from diverse datasets of diverse sizes and shapes.
  • Sound event duration is an important but overlooked factor. Bioacoustic sound events can be as short as 10 ms or as long as 10 s (or more), and most ML systems do not cope well with that scale of variation.
  • How to set up the deep learning training objective? There are lots of ways to make it work — including fine-tuning, prototypical networks, transductive inference, and dynamic weighting. There’s no single best method, so far, and more to explore.
29

Is more about techniques You Tube Video of 10 minutes Dan Stowell | Naturalis
(1) TAISIG Talks: Dan Stowell on Birdsong representations with AI - YouTube
To facilitate collaboration among researchers from different AI related fields, in the TAISIG Talks series, Tilburg University brings together AI experts from various domains to discuss their most recent findings. Each TAISIG talk features three scientists with different backgrounds and at different stages in their careers. More information on TAISIG: https://tilburguniversity.edu/taisig

30

Merlin is an interesting idea, but a disturbing percentage of the user base seems to think that it is incapable of making mistakes, and it regularly does.

31

Well, it was about insects but a very young and small company Aquila made someting for the NDFF database (about 200 million observations) for recognising bats songs. https://www.bij12.nl/actueel/ai-helpt-bij-herkennen-van-vleermuizen-voor-ndff/

People from Aquila also made something for recognising orthoptera songs

32

I did not know this small company already made an bird app, but the app is very sensitive for other sounds and wind. This is 2022 and things wil change i think

https://hbo-kennisbank.nl/details/aereshogeschool:oai:www.greeni.nl:VBS:2:150799

–
I think insects can be hold in a terrarium to identify, photograph and recording sounds

33

Great idea, I have been wondering that as well. I can see a number of big difficulties:
1-Lack of good visual identifications to pair up with the sound recordings–many of the singing insects are difficult to identify to species. Often a specimen in hand is required, and maybe a specialist with a microscope. (On the other hand, some of the usual summer cicadas in SE USA are hard to identify by appearance, but rather easier by song.)
2-Taxonomy may not be clear in some groups.
3-Many insect sounds are very high frequency, and are difficult to record with consumer-level equipment.
4-Most insect songs vary with temperature and this needs to be accounted for. If you look at the Singing Insects of North America recordings, for instance, many record the ambient temperature.
I hope to see something like this in the future, however. Seems like some of the problems could be addressed by more work on recording sounds for insects. (Edit. Coupled with good images!)
As an aside, I will note what looks like a good resource for North America, recently updated:
https://songsofinsects.com/

34

Also see https://sina.orthsoc.org/index.htm

35

It seems i am not allowd to edit previous posts…
https://www.aquila-ecologie.nl/en/vleermuizen-herkennen-met-ai/ (english)

Uiteraard gaan de ontwikkelingen snel en zijn er ook al verschillende opensource projectjes die lopen (zie bijvoorbeeld: https://github.com/rdz-oss/BattyBirdNET-Analyzer ), dus het zal allemaal steeds beter en toegankelijker worden. Maar wat je bijvoorbeeld bij obsidentify zag is dat voor soortgroepen waarvan veel goede afbeeldingen beschikbaar waren en die een relatief uniform voorkomen hebben (nachtvlinders bijvoorbeeld) het model al redelijk snel goed werkte. Bij soortgroepen waar dit minder het geval is (planten en mossen bijvoorbeeld) werkt het nog steeds

en buienradar voor vogels met behulp van het Birdnetmodel en live opnames uit het veld: https://app.birdweather.com/ . Hartstikke leuk en wat het model

https://www.amazon.nl/ACOUSTIC-ECOLOGY- EUROPEAN-BATS-IDENTIFICATION/dp/2366622449 Autheur Barathus
https://www.veldshop.nl/nl/acoustic-ecology-of-european-bats.html

The author acquired unique knowledge and skills over more than three decades of continuous research on bat ultrasonic emissions. In this book he uses here advanced computer-assisted analysis to supplement the auditory approach to ultrasound analysis he initially developed in France. The method described makes it possible to identify about 85 % of bat acoustic records in Europe and to carry out non-invasive bat assessments and in-depth surveys. Thirty-five of the 42 bat species present in Europe are covered. The book also includes access to more than 300 downloadable files online that can be used to practice and develop skills in identifying bats by their sonar signals.

36

As an aside, I think that there may be a time limit on editing some posts though I am not sure what it is/what restrictions may be.

37

@ahospers,

Thank you for continuing to share information about this topic!

Did you see that, four years ago,

@carnifex shared a link to this article in Popular Science? The article is about an app that allows you to identify mosquitoes by the sound of their wings.

The app is called “Abuzz”. You can read about it on the website of Stanford University (USA), here.

You can also watch a short video about the project on Stanford University’s YouTube channel, here.