I saw a tree that just produced thousands of berrries. Dozens of birds of multiple species attacked it and fought over the berries. I took many photos and a video. None of the above were photos of individual organisms. The focus was on multiple species interacting around a single tree. Where, if anywhere, should I post either the pictures or video on INaturalist?
3 Likes
You can post photos for each species, but make it clear which species each observation is for. It helps to crop the images so that the focus of the observation is apparent.
21 Likes
In situations like this Iâll often do something like the following
Same photo, different species, so I edit the photo and circle the target of the observation. You can try duplicating the observation and describing in the text/description which one you mean, but I find that a lot of people donât bother reading, so a visual cue is best.
7 Likes
Respectfully, I would caution against the type of photo anotations linked by @earthknight. While the photos are wonderful and the intent of the observer is evident to a human eye, iNatâs CV has no idea what a red circle around one organism in a larger scene represents. IF such images should ever be included in a training set, just imagine what the non-human Computer Vision algorithm would be trying to associate with the intended taxon! A bunch of gray rocks? The round fish on the left with black and yellow barring? A sceneâany sceneâwith a red circle in it somewhere? Or that little brown fish in the red circle?
Please single out individual organisms by cropping closely to the intended target.
10 Likes
When Iâm unable to crop around a single target (such as here https://www.inaturalist.org/observations/339963873) I write in the notes which organism the observation is for
4 Likes
Respectfully, the entire point of iNat, made clear in the mission statement, is human interaction with nature, not CV/AI interaction.
If you donât know enough to narrow it down, then, keep the initial ID to what you are confident in, which is what youâre supposed to do regardless of whether the CV is engaged or not.
Cropping in tight on small portions of images often results in such a mutilated image that while CV might be able to make some sense of it, itâs not useful to the human eye. It seems counter intuitive, but the broader image not cropped tight is often more âreadableâ for humans.
And your argument about what the CV system gets trained on also kind of falls apart when you consider that unless you isolate and delete the background from all your images there are tones of other living things in each image. Take, as a sort of silly example⌠by your logic photos of a species of butterfly that are obligate feeders of a specific species of plant could have a CV ID issue where the CV system identifies the plant as the butterfly species instead of the butterfly as itâs constantly associated with that plant.
I cannot emphasize enough the point that the CV system is an addition, a sort of crutch to aid in facilitation, but that the core and foundational principle of iNat is to connect humans more closely with nature, not AI systems. And if the CV system makes a mistake, well, the humans correct that based on the information provided by the observer..
I could not possibly disagree with your take more.
And, emphasizing my point about people not reading the description, in the observation linked by @quyksilver they cleary stated which bird the observation was of, and people still chose to identify the wrong one. A circle isnât needed, but an indicator of some type is useful, and arrow, or setting a vignette to darken everything in the image other than the target, etc (although the latter can make the contextual information difficult to make out).
24 Likes
Feel free to post these photos. A photo can show multiple species as long as you indicate which you want identifiers to focus on and also keep up with your notifications so you can clarify if identifiers are confused.
If it will help, circling the subject or adding an arrow are fine. Donât worry about effects of circles, etc., on the CV; itâs programmed to pay attention to commonalities in the photos. Unless everyone puts a red circle around your species in their photos, the CV will ignore it.
Cropping can be a great help. Donât crop so close the subject will be distorted, pixilated. However, simply centering the subject and removing some of the clutter around the edges can help more than you might think.
11 Likes
@earthknight I am totally onboard with the iNaturalist mission statement and your perspective on the role our human observations contribute in that. And I am definitely not an AI or CV advocate or apologist. The issue we face over and over on iNat is that, whether we use it or ignore it or believe it or like it or hate it, CV is an integral part of the current structure for the general population of iNatters to connect with the name of an organism.
I try to think of an analogy for my perspective on CV. Perhaps I view it like a young, ill-trained, disobedient puppy. We canât ignore its presence. For better or worse, it is part of the audience for our human contributions to the platform. IF it is going to have any useful role as a companion for we human iNatters, it needs to be trained properly. CV is already misguiding so many identification decisions worldwide and fauna/flora-wide that it (i.e., itâs failures) contributes significantly to the negative attitudes out in the general scientific community towards iNaturalist. I simply think each of us who are willing should make the effort to make sure the inevitable inputs to CV are useful for future training sets.
7 Likes
To be fair to that guy, they could have seen that I clicked the CV ID for genus Spatula and went with it, since I know some people go through the identifications really fast. I can see how they could have missed my note. As for me, Iâd never seen a hooded merganser in my life so I didnât know that the CV had spat out something completely wrong for what I wanted.
Firstly, human interaction with nature is declaredly one of iNatâs aims, but not itâs entire point. And in any case, the best way to help iNat contribute to said interaction is to help it work at the maximum of effectiveness and efficiency. That means making it as easy as possible for both the CV and the human identifiers to immediately pick out and evaluate the subject of an observation. CV aside, for me personally as an IDer, I prefer not to see a confused image with a multitude of organisms and red arrows, rings or whatever, that oblige me to zoom in and out until I get dizzy. So my personal recommendation would be to crop the master image as many times as it takes to have a single shot, even if tiny, of each organism youâre trying to ID and post this as your first photo in the observation, then include as your second image the photo showing the entire scene. Too much work? Well as they say⌠thereâs no gain without pain 
, and if painâs not your thing, well, maybe there are some interactions between you and nature which are best left private.
I, on the other hand, am quite definitely a proud and absolutely unashamed advocate of this fantastic tool.
As I said, the CV is a tool. It does not make ID decisions, it provides suggestions. Humans, on the other hand, make ID decisions. If they do this by blindly accepting the suggestion made by the CV, then that is because they are using the tool incorrectly and it is they that require better training, rather than the CV (uhm that does sound a bit harsh, perhaps âbetter onboardingâ would tone it down a bit?).
That said, of course the CV can be improved and one of the contributions I try and make to the iNat community is to take and upload diagnostic images which I hope will help to make the CV a more accurate tool in the future.
7 Likes
I suspect that adding disambiguation markings to images is unlikely to have any significant effect on what the CV learns compared to all the other things that might negatively impact its learning â certainly I fail to see how a few images of this sort would make the CV worse than uncropped images or images where the organism of interest is in a corner of the photo.
Remember, the CV is a pattern recognition algorithm only. It doesnât learn what an organism looks like and it has absolutely no understanding of anatomy. It doesnât know what part of the image is relevant for ID. It merely learns what images labelled as that organism look like. If all images in its training set have a certain background, or certain types of observer-added markings, it will learn to associate those elements with that taxon. In fact, this already happens:
It does in fact oversuggest certain species for certain backgrounds where a particular species is closely tied to a particular host (oligolectic bees, leafminers, galls, phytoparasitic fungi, etc.). This is a fairly significant source of wrong CV IDs â and yet the CV training is working as intended.
By contrast, a few photo markups added occasionally and in an unsystematic manner are hardly going to majorly affect what the CV learns as characteristic of a particular taxon.
7 Likes
iNat does not accept videos - but if you host your video elsewhere, then you can link from your obs.
I would also prefer a cropped image first. Then the broad view as a second image to zoom in for more.
Your âsillyâ example frequently happens. CV also prefers to offer âmildewâ for example, even when the âhostâ plant is clearly healthy, thank you.
The iNat Enhancement Suite now lets us crop first, then ask CV - that - blue flower bottom left ?
https://forum.inaturalist.org/t/inaturalist-enhancement-suite-chrome-extension/75349/36
AKA a bad workman blames his tools. It wasnât ME, CV made me do it ;~))
10 Likes
I was in a similar situation. I saw a large mixed flock of migrating shorebirds. The sight was too spectacular not to share.
I loaded to photos and duplicated 5 times. For each, I added a cropped or separate photo of the species and changed the photo order to take the best to the top.
Iâve seen photos with circles and arrows, there are fine with me.
I had to do the same once or twice: Loaded the original followed by the marked up photo and left notes.
4 Likes
I very rarely use markup when needed. (e.g microscopy with several different algae in one photo)
but I usually do what the others suggested:
Ardea alba â âwhite oneâ ¡ iNaturalist
Ardea cinerea â âdark oneâ ¡ iNaturalist
I prefer notes/comments over photo markup, but I donât mind either solution. It often helps to specify the organism you DONT want an ID for, tooâŚ
Agrobacterium radiobacter
I often take photos of galls and bacterial deformations and often need to specify âplease donât ID the treeâ even though I specified the host plant in the notes and observation fields⌠There is always someone not paying attention but thatâs not the observers fault :)
I think, I generally agree with everything earthknight said. Also about the CV confusion. There is always something else in the photos when observing anything (houses, sky, trees)⌠A tiny arrow wont do any harm, the CV will likely be able to figure out what you want :D
2 Likes
I use arrows :)
1 Like
I normally would save a cropped photo for the first photo in the observation and include the original photo as well in the observation.
5 Likes
This CV ID issue is happening because of the non-human patrec blackbox that is CV. Another recent example was an observation of an Ashe juniper tree for which the suggested ID by CV (accepted by the OP) was âGolden-cheeked Warblerâ, undoubtedly because the latter bird species is obligately dependant on that juniper species on its breeding range and images of the warbler in juniper trees are ubiquitous. But there was no bird at all in the mis-IDed original image. That was not an error of the OP, it was a misrepresentation by CV based on faulty learning. CV has learned (erroneously) that âAshe Juniper tree = Golden-cheeked Warblerâ. Hypothetically, and unrealistically, if every prior GCWA image used in the GCWA training set had been closely cropped to the bird, Iâm quite certain the outcome of CVâs training would contain fewer errors of that kind. We canât really look into and tweak the machinations of the machine learning of CV. What we can effect is the quality of inputs.
This is an exceptional case and I concur with those who suggest that red-circled organisms in wide images at present probably represent a minimal contribution to CV training sets. I just wouldnât want to see them proliferate. Better inputs = better CV training.
5 Likes
This was fascinating to read. It seems like it might help if the mobile app allowed zoom on a long press on an area of the image (maybe I can already set it to behave this way?). Long press zoom might resolve some of the instances where an observer took a landscape type image with a small bird in the corner. If they long press the bird before capturing the image, they wouldnât need to crop after or circle anything. Doesnât resolve whatâs described here where the larger scene unfolding is intentionally captured.
1 Like
Another pretty simple option would be to program the CV to prioritize anything within an added circle or in an inset: https://www.inaturalist.org/observations/74716907
This seems like it would be relatively easy to implement addition to functionality.
Regardless, Iâll continue to target any clarification techniques on making it clear for human identifiers. I think the CV system is great, and itâs extremely useful, but thatâs not the focus of the platform.
1 Like
The CV learning to identify a host plant or other consistent environmental feature as the target organism associated with it is a known issue. It can be confusing, especially for those with less experience with the CV. It can be annoying for all of us. However, most of us do learn to distinguish a suggested plant from the bird or insect of interest. I consider it a minor failing, compared to the CVâs usefulness. Fixing it would be good but probably difficult and there are so many other things to fix right now.
4 Likes