Auto-crop images of animals (image computation)

I feel that making the case against cropping entirely would be like saying: never use the zoom lense on your camera.

We need to make somewhat sensible decisions based on preventing image quality loss, and at the same time keeping into account that the context, if any relevant, is still shown in at least one image.

There is indeed the downside that manual cropping consumes a lot of time, hence the need for auto-cropping, with manual human review/adjustments.

I can verify by experiment that, if you do not crop a picture larger than 2048 pixels in any dimension, it will be down-sized (with image quality loss) by iNaturalist during the upload process.

Bump.

You may find more luck asking at a photography editing/tools forum.

begs the question. Why? As an identifier I would prefer not to pick thru 10 pictures, in case there is more info in one of them. Maybe ? Altho insect people have said sometimes The Detail they need is in focus on one of the ‘blurry’ pictures.

25% of IDs from 130 ‘burnt out’ identifiers. Especially the Down South ones during Great Southern Bioblitz!

That same day there was a comment on the obs, asking for the host plant …

I crop almost everything as close to the max 2048 as possible. If it’s something like a crane fly with legs three times its body length, I would just presume that more body details is helpful for an identifiction.

Phone cropping on my Samsung is very easy and quick using the default Gallery picture app in edit mode. I also use the Gallery editor to save the crop as a new image and not overwrite the original.

[Edited] Unfortunately, that is the case for by far the majority of users. That’s the exact reason why the “Needs ID” backlog keeps getting bigger. However, as I see that in the RG observations of yours that I looked at, you provided the initial ID, this indicates that you have a starting point to help with identifying also. When someone agrees with your initial ID, this can bive you confiece in it. You can then look for obervations already IDed as taxa that you know, and then provide either an agreeing or disagreeing second ID.

Hi @bouteloua, which forum do you suggest?

@DianaStuder A very interesting paper “Identifying the identifiers: How iNaturalist facilitates collaborative, research-relevant data generation and why it matters for biodiversity science” which you linked to, thank you.

You said “Why”? Did you mean why I am mostly an observer? Well, for the same reason as you are mostly an identifier. It is what you prefer to do.

According to the paper, we are supposed to work together in harmony. :slight_smile: I quote:

”One of the most essential aspects of iNaturalist is the collaboration between those documenting organisms and those helping to identify them. This collaboration is multifaceted in terms of benefits (Mesaglio and Callaghan 2021, Callaghan et al. 2022).”

You say you don’t like to click through a large amount of images, when you are identifying. But I fully agree with your note on the insects, namely the following:

Although insect people have said sometimes the detail they need is in focus on one of the ‘blurry’ pictures.

In macro-photography, focal lengths are so important, and depending on factors such as the size of the insect, you can not capture the whole insect in focus (the legs might be in focus, but then the head is not, et cetera…). I guess in the ideal world, let’s take an insect as an example, you have 1 image that is super sharp in all focal lengths (which requires an expensive camera which supports focus bracketing → in-camera focus stacking; or specialized software and extra time). Not everyone has access to such a perfect-for-purpose camera. However, even then, there will be sides of the insects captured and others not. And the ones that are not captured might be necessary to properly identify down to a species level. Which is why, it is better to have multiple pictures, from different angles, and/or with the insect having moved its body parts.

For birds it is less necessary, as there are not so many known bird species, and people on iNaturalist just love identifying birds. Anytime you post one, it will be identified super fast, and even if it is quite pixelized (of course not in extreme cases where the bird is too far away / blurry to have enough detail). But, I would think that it’s a very different situation for, for example, many observations of insects. The details might also be important in case some species get re-taxonomized in the future, into multiple species et cetera.

So, I believe that, if an observer provides multiple images, they are supporting the identifiers.

On iNaturalist, I have a journal post with “Rare observations”. You can see how clear the images are. I understand it sometimes even requires dissection of the insects, to see the gentialia, in order to be able to identify the species.

And in exactly the same spirit, I see cropping as necessary (since uploading anything beyond 2048×2048 results in image quality loss) and supporting the identifiers. I believe this is the same reason why @broacher is cropping (see their reply above), and I fully support them on that. My camera is not professional whatsoever, it’s just the standard mini-camera and mini-lenses inside a phone. And thus, my camera already captures little detail, for macro photography. If I would just upload everything without cropping, it would result in even less detail.

But the problem with my camera set-up remains, for some observations, even after cropping. With this kind of camera I can simply not provide enough detail, for many observations to go down to species-level. Of course, it depends on the size of the organism as well.

Anyhow, for now, untill I get a professional camera one day, I feel I need to crop. And the problem is that cropping takes a lot of time. Maybe, for 1 hour of making observations, I need 2 hours of editing. You can see how that eats away a lot of time in my life. And it is not interesting work to me. Which is why it would be great if someone could say if they have a known workflow, for example using Wolfram Image Computation https://www.wolfram.com/language/core-areas/image/ / the function ImageTrim https://reference.wolframcloud.com/language/ref/ImageTrim.html of the Wolfram Language. Because the technology certainly exists. It’s just a matter of piecing it together in a free or not-too expensive way.

It might make some time for someone to answer in that regard (never give up hope), and the problem is that this forum auto-closes posts after 2 months of no activity, for some reason.

Hi @DianaStuder,

That same day there was a comment on the obs, asking for the host plant …

For the case you mentioned, I think it’s regarding observation 324867586 (I’ll not link to it as I think it’s a policy on this forum not to link to your own observations), it is still not identified, even though I made a new observation 325672838 for the plant underneath the individual after I had a request about the plant from the former observation (such requests are extremely rare … but nevertheless I hold the observations on my phone for roughly 30 days after uploading, so if anything might be requested I can re-visit the original video footage and make 20 extra crops and extracts of the surrounding species that might be of interest).

Such splitting of observations per individual is also in line with policies of iNaturalist, as it of course can also inform the “Expected Nearby” modelisation of our CV.

A side-note, but regarding ‘Observation Fields’, I am not sure what ‘observation field’ I should add to the former observation, to structurally link it to the latter observation. Because at this point, it is not clear that the former is an arthropod / insect / plant part / …?

As for the leaf galls, I discussed this in a former reply in this post: what to crop away and what not requires somewhat of a learning curve, for certain organisms; which is where observers can learn from identifiers through various communication functions offered by iNaturalist. But for I would say 95% of my observations, or more, any surroundings are not going to bring down the observation to a species level. Please let me know if you think that, for the largest majority, if you had a very wide-angle picture of the environment, whether or not you could bring them down to species level. I think, for the very large majority, it’s not possible. Quite the contrary: my observations need more detail on the individual, which is lacking due to the image quality.

Hence I think we don’t need to use these rare examples to try to generalize and say: don’t crop, never use your optic lense to zoom in when taking a picture, stand away as far as possible from the arthropod and take a 360 degrees fisheye picture of your whole environment. That just wouldn’t give any proper species-level identification for most arthropods.

My point here is just that, for example for my observations, a picture of the surrounding species is usually not very useful at all in order to bring the observation down to a species-level. I see better image quality as an extremely more important factor. Surrounding species can be instead other observations, as per iNaturalist’s policy. And where necessary one picture could be retained with less or no crop.

Of course, I am also not saying that there are no cases (and depending on the types of obervations, there can be many cases) where surrounding species are relevant. That’s not what I am saying.

To summarize, it’s not because for some or many observations, it’s good to have wide pictures that show surrounding species, that there is not also a need for an auto-cropping tool, for some or many observations and/or pictures within 1 single observation.

This may be lengthy but I feel that, for many (most of my) observations, cropping is better, due to the pixel limtation by iNaturalist. That is different from saying: every picture needs to be cropped (which I am not saying… so where useful uncropped pictures could be retained, and they can split into multiple observations). But sometimes I have e.g. 100 observations on my phone, times let’s say 10 pictures per observations, for which the plant the insect is sitting on, is not crucial to bring them down to a species-level, that’s 1000 images to manually crop, and I feel I waste my life doing so as it could be automated with the right tools.

While the remarks which you and others invoked here are interesting and meaningful, I remain looking forward to experience or pointers on such tools.

if you’re still looking for pointers, i feel like you didn’t even bother to follow this lead / pointer.

I hear you - it does take time to crop photos - and I choose to do that myself. For me it is plants not insects that I observe. And the macro details, which my photography skills struggle with, are important. But then I also choose to upload few obs, and am willing to give each of my obs the time it needs.

Why do I encourage enthusiastic observers to make some IDs for others, to pay it forwards and show their appreciation for the IDs they expect and receive? It is only when you apply your own knowledge and skills to make IDs, that you will realise why we have preferences in turn. On iNat there is always a ‘friendly’ tug of war between identifiers and observers. Both sides are busy and expect The Other to do the ‘more’ work.

I have a backlog of my own obs for most of this year. I would prefer to get to those, with each month that passes it is harder for me to work out exactly where on the trail that one was.

PS you may upload the link to your relevant obs. You may not call out another iNatter for bad behaviour, and you may not ask for IDs in the forum.

this is true for any camera.

you are under no obligation to post observations of a certain quality. if cropping so many images is too tedious, crop less of them - or don’t bother cropping.

if people comment asking for a host plant ID and you have a separate observation for the plant, you should respond with a link to the observation of the relevant plant. it seems instead you tagged the top IDer and then disregarded their response to you.

Hi @dendroleon, I hope you are doing well and thank you for your comments.

this is true for any camera.

I do not understand your reasoning here.

If cropping so many images is too tedious, crop less of them - or don’t bother cropping.

Completely agree that nobody is forcing nobody to do anything on iNaturalist, unless one is getting paid maybe! The goal of the current forum post is to ask for a way to auto-crop, for example animals. If one would like to question cropping alltogether, there are many other posts on that topic, for example: https://forum.inaturalist.org/t/a-plea-to-crop-your-photos/35251 But let us try to keep the current post for auto-cropping, thus not to be anti-cropping.

If people comment asking for a host plant ID and you have a separate observation for the plant, you should respond with a link to the observation of the relevant plant.

Yes, I agree with you, that seems to be in line with the official policies. That sort of linking is what I aim to do, for the 2 or 3 or so observations of my 4700+ observations where someone asked for the host plant / the plant the arthropod is resting on. If I don’t have the observation, I aim to create / duplicate the observation for the identification of the plant, see e.g. https://www.inaturalist.org/observations/324867586 → https://www.inaturalist.org/observations/325672838 .

If you’re still looking for pointers, i feel like you didn’t even bother to follow this lead / pointer.

Hi @pisum, thanks for your comment. The link https://forum.inaturalist.org/t/uploading-ai-bounded-images/50884/10 you mentioned regards object detection, correct? It refers to https://github.com/agentmorris/MegaDetector , which in turn points towards a neat tool to test it on Huggin Face https://huggingface.co/spaces/ai-for-good-lab/pytorch-wildlife , but it is not entirely clear to me how to connect it to a cropping functionality.

Such an open source code to do object detection could be one of the many puzzle pieces, certainly. Object detection would be the first step. Thanks, it is helpful and good to know of other posts in a similar area. But as you said previous, there are lots of ways to do object detection. The https://reference.wolframcloud.com/language/ref/ImageTrim.html which I mentioned also crops to a region of interest, so it could be another step to one day have a complete flow, or understand how to use such a function.

So I feel still it would be great to get some pointers of someone who succesfully set up a whole flow, or to hear of a complete tool to do auto-cropping. So far I have heard none. Summarizing, people mostly just love to come here and say “don’t crop” instead. But that’s fine, becaue it bumps the post, and that way perhaps someone who knows more about auto-cropping will one day reply their lessons learned from experience.

As for the Wolfram ‘ImageTrim’ functionality which I posted near the beginning of this forum post, when I find some time, I might look into whether or not it can be used with the Free Tier of Wolfram Cloud, or Wolfarm Engine + WolframScript. It might be exactly what I needed in the first place, but I have never used these environments before so it will require some experimentation and time to find out if I can, and to see if the flow can be smooth and save time.

Lastly, just out of interest, I am not sure whether the Computer Vision model of iNaturalist itself in the background also uses such a object detection + auto-trimming functionality? If we’d know who maintains it we could ask, they probably have ample experience in the field to give us some answer on the current post in general. It’s not clear to me from https://www.inaturalist.org/pages/team

look at the first point in the “How do I get started with MegaDetector?” section of your referenced Github repo.

Thanks @pisum, as I am going through it now I am very grateful to learn about the proposed GUI: https://addaxdatascience.com/addaxai/ Looks very user-friendly and I will do an initial test in the near future and report on preliminary results!

@pisum I just tried running AddaxAI v6.23 on simple pictures of birds (Advanced mode — MegaDetector 1000 Redwood — Model to identify animals: None — Post-processing: Crop detections), and the resulting crops didn’t show ANY bird! So far so bad.

I am now downloading the model “Global - SpeciesNet - Google” (it seems to be the only model relevant for Paraguay at the moment) to try that way, hopefully that will produce better results. The download is failing currently due to shabby internet connection, so I’ll might have to continue that another day.

Is there someone here who was able to use AddaxAI (e.g. with MegaDetector) succesfully to auto-crop?

just for animal detection, you only need the MegaDetector. you don’t need a species identification model. AddaxAI ships with some test images. see if you can get crops of those test images. if not, you’re doing something wrong. if the test images process fine, but your images don’t, then you could try using different images or adjusting confidence thresholds for post processing.

plenty of people use MegaDetector / AddaxAI to detect animals in images just fine. if you have images of obvious birds, i’d be surprised that it wouldn’t be able to detect them. if your images are partially obscured birds or are very blurry or have lots of noise that look like animals, you just have to realize that it may not always be possible to detect every partial or blurry bird in an image and also that noise may be mistaken for animals.

if you can identify particular images that AddaxAI isn’t handling well for some reason, you could take those to the AddaxAI forum, and see if folks there can provide additional insight.