Annotations on photos, such as a circle or arrow pointing to a subject or area of interest (although we recommend avoiding them)
Are such kinds of mark-up discouraged because they would train the computer vision model to look for geometric shapes in solid colours? (I see that single photos are preferred over collages, perhaps for comparable reasons.) If so, how feasible would it be to allow users to set a flag to exclude an image from the CV training database?
I’m asking because I was really struck by the 3-D geometry of an orb weaver’s construction of its web, as it pulled the radial “spokes” forwards or backwards from the plane of the web. So far, though, I’ve posted only the unmarked versions of the photos.
I cannot give you the answer you want but I share your assumption about the CV.
As to my understanding, the CV is only trained on the 1st photo of each observation (read somewhere on the forum), which means that adding markup on a 2nd image could prevent issues with that. But there are probably enough other good reasons.
I do use it, since it’s allowed but the cleanest approach is to embed the image with markup in your ID or comment. That requires uploading it elsewhere first, tho. I generally do this with habitat shots, follow-up shots (technically could be new observation of same specimen), images I am drawing on for explenation and for spectrograms. (I use imgur, because it’s the least complicated solution)
the CV can be trained on any photo from a given observation. What you’re thinking of is, when you upload or open an observation in a browser and click into the species ID bar to see the CV suggestions, those suggestions are based only on the CV looking at the 1st photo of that particular observation
Oh, that makes so much more sense. Subconsciously, I’ve been using the CV on each photo individually before combining them into a single observation for exactly that reason (to have several chances on a match). My brain just didn’t make the connection. Many thanks for the clarification.
I don’t visually annotate observations, simply because it’s too much work, but this reminds of a written guide for amphibians I saw recently. Each organism in the guide was accompanied by a photo, and each photo had one or more arrows pointing to diagnostic character(s) for that organism. A very powerful visual tool for identifiers!
personal take: I think such annotations or marks are not going to hamper iNat CV model as is, the minimum photo count to train a model could be able to offset those spurious one-off things of annotations just as one off background or scale or any such background things in any other observation is. But ofc if every observation of that species or inat as whole follows that analogy and everyone makes such annotations - that’s another thing and is worth like old fable stories (which is also why its not recommeneded by default to all) - recently i noticed people annotating their photos with direct species name (which can be faulty and is different from community consensus - uh oh), or some even annotating with GPS apps spoofing banner at bottom to cheat assignments oof.
But if you think there is a diagnostic thing for that species, i love people annotating those things, or collages of different lifestages and then adding them as atleast one iNat taxon page photo. It has helped me in dozen cases as of now when I encountered them first time on iNat instead of trawling through all comments (yeah lets see later what new inat comment model will become) or elsewhere (especially some caterpillars),
And in your case of web annotation it could be important point to highlight and can be added readily into observation (who knows maybe that pattern is actually diagnostic to some species/genera that actually helps CV to learn and focus too better) or maybe it will catch researcher eye someday who is casually going through their GBIF imports (also please educate newcomers of iNat licenses when you see all rights reserved on rare species) and form new hypothesis. I anyway keep best cropped photo as first (app has both options) and then all others with any extra details next and notes
Agreed, training the CV on a few images with arrow or circles around a beetle is no different than training it on photos of the same beetle sitting on 50 different backgrounds. Yes, the CV is pattern-matching and taking the whole photo into consideration, but if extra “noise” in the photo besides the target organism were going to ruin the CV, then it wouldn’t work at all in the first place. The only way I could see it being a problem for the CV training is if one particular species were consistently uploaded with circles drawn around it while similar species were not, it may learn to recognize that circles = that species. So, I guess don’t systematically draw a circle around every Alder Flycatcher you upload and skip the circle on every Willow Flycatcher. I don’t expect a situation like that would be very common though.
I can only imagine this happening if the observations of a certain species are dominated by one observer who habitually adds mark-up. Which is possible with more obscure species that are only just eligible for CV training, although I have trouble imagining someone having the patience to add mark-up to 30+ observations of the same species.
I use iNaturalist to host the images, with the corresponding observation being where I upload the photos. That way the photos are still accurately describing a species in its observation. I always have the edited photos occurring last in the observation’s photo set. Link to observation.
I’m sure the impact to the CV is slim to none. Besides, I figure the usefulness of my guide outweighs any negative impact edited photos may have.
This is amazing. Something like this might make me not hate trying to photograph and ID plants. Better yet, could you just move to Pennsylvania and personally handle all my plant observations? It’s very nice here, I’m sure you’d like it.
Short of that, is there any central collection of these guide-images, or a particular way to find them among your observations?
I didn’t realise markups were discouraged on iNat. Thanks for the tip. Sometimes I will put a coloured circle around something that is not immediately visible in a photograph e.g a small bird in dense foliage. I do this to save time for potential identifiers.
A better solution to focus attention on a small or difficult-to-locate creature–and to help CV and us IDers–is to crop the image close to the subject. It’s my mantra for every iNatter. It’s amazing how much improvement in CV suggestions can be derived from a well-cropped image. No need for circles and arrows and it’s probably quicker to accomplish.
cyclosa actively adjusted tension in radial threads by pulling them under anticipation of prey in those directions from past experiences, because that higher tension tuning translates to better vibration transmissions for those radial lines and efficient prey capture in turn.
now, another caveat is that these radial threads are the main force carriers in damping prey kinetic energy to thermal dissipation - Spider orb webs rely on radial threads to absorb prey kinetic energy - so its a careful balance on when to tune and how long while not overly stressing those threads to break.
I think this has sort of been raised (didn’t read the entire thread) but if someone can draw an arrow or a circle on a picture they might as well just crop it instead! Though I realize there may be some interest in an overall photo highlighting the subject as well.
