I’m wondering if images get added to the training model automatically or are they hand picked by iNaturalist staff?
I’ve seen observations where users have uploaded photographs of tiny bugs. They then upload an image within the same observation of the habitat / landscape they were found in. The habitat image does not show the tiny insect at all.
I’m wondering if this is bad practice or if this is fine for the AI? I’m also wondering if enough people started doing this would it pollute the AI’s training model with irrelevant images and therefore reduce it’s accuracy at giving correct ID’s?
Photos are not hand-picked, so staff have discouraged adding habitat photos in the past for this very reason. Although I don’t think it’s a strict rule.
I can only think of one Observation where I have included a photo without the organism: this one, wherein the images act as a sort of fotonovela and the final photo finishes the story and shows the extent of the insect’s action.
A habitat image might be a useful determinator for some species – I have definitely had Identifiers ask questions about habitat (plant, where on plant, weather, other species nearby, etc) – so it might be a worthwhile tradeoff in some instances?
The best way (in my opinion) is to add a photo of the habitat in a new observation, identify the dominant organisms as the subject of the new observation, and link the observations using observation fields or links in the description.
It’s not something that staff want people to be doing. All habitat photos are problematic for CV training, but those that also show the organism are allowed anyway. What you describe is not permitted by the observation guidelines, since all photos should include evidence of the observation’s subject.
A practical downside to doing this is that, when someone wants to display all photos of a species or other taxon, the system shows any and all habitat photos (or photos of other non-target organisms) all together in the same collection, without any indication of which ones are linked in the same observation. This can be confusing, and create an impression that gross misidentifications are happening at a higher rate than they actually are.
Another reason why they want each and every photo in an observation to show the actual organism.
Personally I do take habitat shots with my observations, but I always make sure that the target organism is visible in the shot. If it is small, I will point out its location in the description or a comment.
I absolutely see all the excellent reasons why adding habitat photos is discouraged, but for IDing purposes it could be SO useful, particularly in the case of plants (where I do most of my IDing). I’m just wondering… would it be beneficial and/or technically feasible to provide an additional field in the “notes” section for uploading general habitat photos? Or to encourage inclusion of habitat photos in the comments?
I’ve wondered before whether an option to ‘exclude photo(s) from CV’ would be workable/helpful. There are times when a photo is (a) showing little enough of the organism or enough of another organism that that it’s not helpful or (b) showing context rather than focusing on the organism - and while they can at times be okay or even helpful for ID, they’re far from helpful in terms of training the CV. We encourage people to make lots of observations; being able to exclude particularly poor quality photos might make the system work better for everyone?
We need a better solution for habitat photos. If the habitat photo is ordered after a few that show the organism? A new annotation for Habitat / Vegetation type?
And I would still like to be able to annotate single photos.
Look for pictures of the fruit of … scroll thru gazillion to find the pictures which do in fact SHOW fruit.
With Google Lens, I can crop an iNat photo, and ask for … that bug, that fruit. It is, in the moment to get to a possible ID and does not affect the original picture.
That function would also be useful on iNat. Especially when the obs is for - the bug walking on the pavement, but the picture is a nice portrait of Little Boy Intently Studying (some blurry bug). Or a group of people and one is holding the Relevant Something.
To be clear: I am not in favor of habitat shots in which the focal organism is not present (i.e., a landscape where the plant or bird is only a tiny portion of the scene but still recognizably somewhere in the photo is OK, but a photo where the bird has flown away would be questionable, unless it has left behind some other visible traces of its presence – a nest, pecked holes in a tree, etc).
But…I don’t think a concern about potential effects on CV training is a good reason to exclude either habitat photos or poor quality photos. From what I understand, the CV looks for resemblances/common visual elements in photos; if there are individual photos in its training set that do not fit the pattern it will consider these photos less relevant. So occasional habitat photos will likely be treated as outliers – or they may be unexpectedly relevant if a species is strongly associated with a unique sort of habitat. From the CV’s perspective, there is probably not much difference between a habitat photo where the focal organism is not present and a photo of the organism that includes a lot of context/habitat and the organism takes up only a minute portion of the photo.
Excluding poor photos from the CV also seems like it would actually make the CV less useful for one of its primary purposes – helping observers figure out what they saw. This is not identical to a goal of training the CV to distinguish organisms per se. If that were the goal, no doubt this could be achieved more effectively by using only photos carefully selected based on the clarity of the image and visibility of important characteristics for helping distinguish species A from lookalike species B; such a project might also include standardized photos of specimens for better comparison.
Except that most user photos are not perfect photos that are cropped to highlight the focal organism and clearly show all the relevant features. Rather, they are photos taken with cell phones from some distance, often neither completely in focus nor cropped. The CV will only be able to make sense of such photos if it is trained on photos that are representative of what the average user will be submitting.
For some reason it put the habitat photo first, so now that is the only photo of those two that you can see on the gbif species page. I don’t know how to fix this (deleting the photo on iNat didn’t solve it)
Perhaps we should be able to mark individual photos as ‘habitat’, and it adds the word ‘habitat’ over the photo in the corner or something. Then it can be excluded from CV (much as I agree with @spiphany’s points above), and at least makes it clear why such a picture is there if it ends up out of context.
I agree completely. If the CV can’t cope with all the types of photos that people upload to depict their observation, then it’s failing at its task! And if humans are expected to change their behaviour in order to satisfy the algorithm, then I’d start questioning its value altogether. Who is the target population that inat is supposed to serve? Its human users, or the computer vision system?
I have previously raised the issue of including habitat photos and am strongly in favour. They are both useful for identification and interesting. They definitely support the iNaturalist aim of getting people to take more interest in nature, so if they are a problem for the secondary aim of training AI, I urge a work-around to be implemented.
Fair point. I suppose I’m thinking particularly of cases where observers will take a photo of a particular plant, notice that there’s a fragment of another plant in the corner that happens to also be probably recognisable, and duplicate the observation for that. In fact, at times I’ve almost corrected the ID before noticing a note from the observer that both species are present, prompting me to look more closely. I’m far from convinced that including such photos in the CV is anything but misleading.
Okayish according to iNaturalist’s help page (“should be depicted”) and staff on forums. Sometimes hard to tell whether a picture is deliberately of the inferred habitat, or rather user error/app bug having uploaded unrelated pictures - better ask the observer, if not readily mentioned or obvious.