New DQA: Does "Evidence related to a single subject" mean the same individual is in all photos?

I agree with others (@anon83178471 @silversea_starsong @sedgequeen @pisum @annkatrinrose) that using the new DQA in a broad way to flag observations where there are multiple individuals of the same species would be problematic. For one, these types of observations have always been allowed to be IDed to species level on iNat previously, and there are a LOT of existing observations like this (including forum users who’ve attested that they have many). If the DQA field can be applied in this manner, I would guess that there are 100s of thousands of past observations it could be applied to.

Many observations show photos of groups of organisms without a singular clear focal individual, and are already challenging to annotate for this reason.
Some examples of problematic observations would be:

  • flocks of birds or fish (with different sexes and life stages)
  • large patches of plants (with potentially different sexes and life stages)
  • groups of fungi where we can’t tell where one individual ends/another begins

I’m sure that there are more scenarios, but many iNat observations pose challenges for annotation. To me, the loss of a valid species level observation to Casual because it can’t be annotated is not a good cost/benefit tradeoff - I’d much rather have all the observations that can be IDed to species with some annotated as RG rather than a more limited subset of observations that can be annotated.

In these types of potentially multi-individual observations, and even observations with multiple pics of single organisms, it can be very difficult to determine if the same individual is included in all photos, as this could rely on needing highly detailed pictures or high familiarity with the species. This means that applying the DQA consistently in this manner will be very difficult and could lead to conflict/disagreement. It could be very frustrating for new users. It could also end up taking a lot of identifier time, leaving comments to try to get observers to split up photos into multiple observations when one of the key reasons for the creation was to save IDers time.

I think that the goal of increasing the annotatability of iNat observations is a great one, and that allowing tagging of focal organisms on pics sounds awesome. But I think using this DQA field to downgrade multi-individual observations because they can’t be annotated effectively in some cases is getting a little bit ahead of the goal here and would overall be a negative change to iNat.

33 Likes

I won’t enjoy it, but I also wouldn’t mind much.

OTOH I frequently see one image with an adult and one with with a larva.

  • sometimes they are from the same species and I can identify both - fine
  • sometimes the observer thought they were from the same species but they aren’t, not fine, but covered now
  • sometimes the observer thought they are from one species but I don’t know, since I can ID one but not the other - annoying

As an observer I try to follow the 1 obs = 1 individual pretty strictly, apart from having images with multiple (to me) identically looking ones. And if someone points out to me they are not all the same I will make a decision about which one it is.

As an IDer I’d be happy if everyone would be doing the same.

Still never gonna happen I guess. :-)

7 Likes

it’s covered now, yes, but in the old system, and identifier would leave a comment, and i would see it and be able to correct the observation appropriately.

in this new system, it’s possible the identifier just marks the observation as multi-subject without any explanation, and i never get notified, and i even when i figure out that the observation is marked as mutli-subject, i have to guess at why the identifier thought it was multi-subject – is it because there are multiple individuals? or is it because of multiple species?

6 Likes

Ah, yes. I guess this situation doesn’t really happen with photos of animals.

That being said, it’s really only relevant to #2.

Is still valid.

1 Like

Staff promised years ago that after the notification system revamp, we would get notifications for changes to DQA, but they stopped working on the system revamp…

8 Likes

Agreed. Personally I will not be using it as long as all photos contain at least one member of a single species.

9 Likes

I regularly create 2 observations where for example a spider has caught an insect in it’s web where both animals are together and impossible to crop to a single species (not to mention significant scientific value in showing animal and prey wholistically). I add a comment to each obs saying which it’s for i.e. the spider or the insect in my example.

The way this is worded both obs would end up as casual the way this DQA is worded even though both species can be perfectly identified to RG taking into account the comment.

Needs to be reworded to only handle the clearly wrong cases where multiple photos show unrelated content

4 Likes

This step alone should easily take care of any ambiguity, without needing to reword the DQA.

3 Likes

Or several different organisms of the same species (e.g. photo 1 of individual box turtle 1 and photo 2 of a separate individual box turtle 2 which may have a different age or sex etc).
Should both be flagged as not having a single subject and the observer should be encouraged to separate

Does this mean that every moss observation on iNaturalist should be marked as Casual? What about entomopathogenic fungi? There’s no practical way to separate the individual organisms in such observations. I think @cthawley makes some good points that should be considered, and I also agree with @sedgequeen that the wording needs to be improved or at least the guidelines for how to use it.

10 Likes

i’m not sure about this. suppose observation A has a single photo of both a male bird and its female partner, and suppose observation B has the male as one photo and the female as a second photo.

observation B falls into loarie’s second case of things that should have the DQA item marked thumbs down.

but in the current world, if the observer didn’t annotate either A or B as male or female, would you as an identifier try to annotate either A or B?

2 Likes

Yes. I likely wouldn’t bother for common species, but if someone posted a photo of an adult Gray-headed Swamphen standing over its nest with the eggs clearly visible. I would annotate for the eggs and post a comment along the lines of:

“Great observation. I’ve annotated this observation’s life stage as “egg” because iNat doesn’t have any reference photos of this life stage for this species. However, if the adult is strictly your intended individual of focus, feel free to tag me and/or remove/change the annotation.”

I want to clarify, that I would not personally use the DQA for situations such as this. My main point has always been:

3 Likes

just for some context, i think this is where iNat is trying to go in the future with computer vision: https://github.com/visipedia/inat_loc.

i think they want to be able to pick out where the organism is in the photo, and i think a mechanism to delineate where particular organism or a particular annotation exists in a photo is actually the solution for multiple individuals and differing annotatable features, etc. but the delineation doesn’t have to be made by computer vision. it would be great if regular folks could make those delineations by drawing boxes on the different photos.

3 Likes

I strongly feel like this is a bad stance to take. Every individual even of the same species within an observation now must be separated or be relegated to casual? There are so many groups of insects where this makes zero sense, shall I somehow try to crop down that ball of hundreds of aphids into individual obs? How about ants? How about for the few identifiers most of these types of animals have whose queues have now been potentially magnified exponentially? Also seems to somewhat muddy the overlap of situations when there are interactions between different species, but the species intended to be identified is made clear in the comments (predator/prey interaction, hempitera standing on coleoptera, interesting photos that show actual interactive behaviours?).

I could say a lot more, but I feel like the point I’m getting at is clear.

10 Likes

Multiple orgsnisms in a photo are fine. But a observation can only have one individual organism as it’s subject (eg a single male adult bird somewhere in the flock) and all the photos should relate to that subject (eg if photo one is a flock of birds, and the description, ids and annotations all indicate it’s the subject is an adult male in the flock, photo 2 shouldn’t be a photo that only contains a different juvenile female subject regardless of whether it’s the same species of bird or not. But I f photo 2 also contained the adult male it would be related to the subject.

Here’s the GitHub discussion about the scope of the feature

https://github.com/inaturalist/inaturalist/pull/4013

Note, this is not intended to be:

  1. a way to salvage the data (e.g. a way for the community to force the subject to be the first photo and indicate that all other photos should be ignored)
  2. a tool for flagging controversial practices like habitat photos or bat sonograms that DO relate to the same subject
  3. a tool for resolving ambiguous subject situations - e.g. observer posts a photo of a mallard and a pigeon and doesn’t indicate through an ID or description what the subject is
6 Likes

i understand why you all implemented the DQA item to solve for multiple species situations, but why would you all try use this to solve for multiple individuals?

i get that different individuals can be annotated differently, but as i noted in my flowering plant example, even within an individual different photos can be annotated differently. so if the annotation problem is the problem you were trying to solve by applying the DQA to multiple individuals, i don’t think you solved the problem right.

13 Likes

Can you point me to the flowering plant example?

The individual plant is still flowering even if not every photo shows the flower. Annotations are used for more than just photo references (e.g., phenology).

That being said, photo references are an important benefit of annotations and it does tend to be messier with plants than with animals when annotations are for the entire observation instead of specific photos.

I think something like this could maybe work, but it would drastically change the way iNaturalist works on a fundamental level.

Do you mean something like: adult male Bufflehead in the blue box pictured with adult female Gadwall in the red box (both individuals in the same photo, a single observation for both species)? I don’t see something like this getting implemented.

here’s an actual observation that has that situation: https://www.inaturalist.org/observations/85959265. it’s annotated as both flowering and fruiting. the second photo has fruit, but the rest of the photos show flowers.

1 Like

Right, but all the annotations are correct for that individual because that individual is flowering, fruiting, and flower budding. (Though it should not be annotated as female because Trumpet Vine is monoecious.) Annotations relate to the whole individual, not just a part of the individual.

If I took a photo of a plant in bloom but didn’t include the flower in the photo, I shouldn’t annotate the plant as “no evidence of flowering” because that is incorrect phenological data.

2 Likes

I think this is a profoundly bad idea. I’d like to use much more colourful language to describe it, but I will refrain.

Not only is it a bad idea on it’s own, but it sets a very bad precedent. In trying to address a relatively minor issue, you’re creating a much larger one. I already have a hard time motivating myself to continue to spend countless hours IDing observations. This is going to make the task that much more onerous.

This is precisely the kind of scenario I had in mind when I argued that we should maintain the convention where a person’s contributions are wiped out when they delete their iNat account (hint hint).

8 Likes