I’m curious whether it’s particularly useful to upload photos of organisms that are already included in the CV, but using photos that fool it (or return inconclusive results). The photos would be conclusive enough to make it to RG.
In other words, does the CV use photos that fool it to improve during the re-training?
CV uses photos from various people, with different cameras. Yours would be part of the mix.
Do you intend to upload ‘trick’ photos? You may irritate identifiers. Or be swept into - Human - Casual - Next.
No, I didn’t mean uploading trick photos. Sometimes you catch an animal at an odd angle…something that fools the machine, but a human can easily determine.
My main curiosity is whether the CV re-training finds photos that it ‘missed’ to be particularly useful , and whether organisms that are already in CV get re-trained at all?
No that is definitely useful. Tiwane had an earlier comment - along the lines of using the ‘real’ photos of what people see and snap. On today’s hike one of our common and familiar flowers confused us - something about the angle made it look quite other - what IS that?
Yes weird angles and such can for sure help the CV. They can help force it to learn actual features of the organism instead of just learning to identify the photos people typically take of the organism. I anticipate that future CV models may eventually benefit from deliberately prioritizing diversity of angles and appearances such instead of just picking them randomly, although this may be somewhat difficult to figure out how to implement in practice at present.
Edit: currently the CV doesn’t specifically look for photos that fool it. In the current implementation and dataset that would be risky because some photos should ‘fool’ it; for example the kind of observations that appear in this project: https://www.inaturalist.org/projects/ignore-the-elephant-seal. The CV takes into account that some photos should fool it by adding a softening term in the training that reduces how much it gets penalized for getting things wrong in some cases.
As I understand it, go ahead and upload photos from odd angles or with odd lighting or unusual plumages, etc. Those photos then have as much chance as any other of being used for training. The CV will take care of itself. The CV will, in effect, ignore photos that are too odd or useless. (Not quite, but they’ll be “outvoted” by the others.) The CV can, apparently, learn two or more plumages, etc., for a species, so if your odd angle is repeated by lots of observers, the CV will add that look to its “ideas” about what images that should be labeled as that species.
Can’t find what I want but here are some iNat blog posts about CV. Fascinates me!
https://www.inaturalist.org/blog/54236-new-computer-vision-model
https://www.inaturalist.org/blog/59122-new-vision-model-training-started
https://www.inaturalist.org/blog/69958-a-new-computer-vision-model-including-4-717-new-taxa
ignoring the needs of computer vision, i think its helpful for humans to have reference photos of all sorts. things like distant birds in flight or whole trees or extreme closeups of scales, etc., will all be useful to someone at some point.
It partly depends on how many photos of that taxon the model has to choose from when its trained. Only 1000 photos per taxon are used to train the model, so if you have a weird shot of a a taxon for which there are more than 1000 photos, then your photo may not be chosen when the model is trained. If there are fewer than 1000 photos, then your photo should be used for training. Whether one weird photo makes the difference there, I don’t know.
I think this is a better way to think about it.
Thanks for this question.
I’ve been purposefully uploading dorsal shots of moths as my first photo (if in a series of photos) . I also try to tag my own uploads “ventral.” When I identify for others, I often have to slog through dozens or more observations to find one that confirms an ID from an unusual vantage point. It will be wonderful when CV learns to ID from all vantage points.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.