AKA as must reach 67% agreement. Please form an orderly queue … @mention … we WILL get there, one day.
Here is the latest example, where 16 identifiers won (I could, finally, Withdraw my Support against that unhelpful algorithm). 16, where 14 could have IDed 7 more obs. Or at least 6 more if you accepted my logic (2 agree, 2 against, another 2 agree = CID)
If two agree is the ground rule.
Then surely we could be reasonable and add one extra right to balance each wrong - on top of the basic 2 agree.
For birds I don’t care. But there is a substantial part of iNat where it is impossible to find enough identifiers to convince the algorithm. Which is definitely set up with a home advantage against identifiers.
On a forum when not everyone comes from the same background and has the same issues in mind, I would kindly suggest including a little clearer explanation as to what the topic is about. It took me a pretty long time to understand what you had in mind here from the post.
What I did not understand though is why did you withdraw the ID. Were you or were you not able to ID the bird into the hybrid? If yes, why would you withdraw the ID - if no, why would you put the ID in the first place?
I do think that reaching a threshold of exactly two-thirds should be sufficient to generate a consensus taxon – there are plenty of observations where two initial identifiers were wrong (happens to the best of us) or acted spuriously (i.e. put all their faith in the computer vision, which ought not to happen), and subsequently a full four identifiers have attempted to correct it… only to need to hold out for an elusive fifth identifier, in pursuit of that more-than-two-thirds threshold. However, any threshold will ultimately be arbitrary, and I doubt my opinion will get any more traction than the status quo, because it is just an opinion more or less.
I’m not sure what the proposal in the topic-starting post here is. Is it that the numerical threshold should be instead that “X for + X against” would balance out to no net ID, and then 2 IDs for any taxon on top of that “net null” would bring it to research grade for whatever the 2 additional IDs are? If so, I’m also not sure how that would be implemented, except in the subset of cases where it is the same proposal as mine above (not to self-aggrandise… I just believe that an exact-two-thirds rule is numerically simpler, unless I’m fundamentally misunderstanding the topic).
ID1: Species A (wrong)
ID2: Species A again Observation is now incorrectly RG.
ID3: Species B (right) Observation is no longer RG under the current system.
With your proposal, a fourth identifier is required just to get the observation out of RG, which is unlikely to happen unless someone is tagged in by IDer #3. There are definitely tradeoffs involved, but I think it’s better to have more observations in Needs-ID limbo than to have more observations incorrectly RG.
perhaps a threshold of total identifications should be applied such that it’s more than the numerically minimal two-thirds (a mere two against one can’t count as research grade)… but then that complicates the original proposal when it was supposed to be simpler…
I can see why the system may have been implemented in the current form!
In practice, I think the community taxon/RG status algorithm is imperfect but mostly fine; as @pisum persuasively argues with the ‘ungrafted taxon’ hypothetical in that thread, any plausible alternative algorithm also has weird edge cases.
I have also actually seen cases where an observers initial ID actually was correct but an expert came by and added an incorrect disagreeing ID (i.e., perhaps they were going through and cleaning up 500+ near-identical CV-related mis-IDs and just went too fast/overzealous/too late at night/misclicked), and then another experienced user agreed with the expert. Experts are usually more likely than a random observer to fix an obvious mis-ID if you tag them of course. So while I think it is reasonable to consider the expertise of an existing disagreeing ID as a factor in your mental decision-making on whether to agree with them or not, it shouldn’t be the only factor.
Part of the reason is that not all disagreements are created equal. For example, I think it is often appropriate to more or less blindly agree with experts in a case where you are certain the original ID was wrong but would otherwise be uncertain what it was specifically. For example, say a user had originally ID’d something I’m certain is a plant as a sea anemone, and then two plant experts have ID’d it as anemone multifida var saxicola. In that case, I’d feel pretty comfortable just agreeing with the experts, because the initial ID should just be made irrelevant. On the other hand, if the observer had ID’d it as anemone multifida var. multifida and two experts ID’d it as anemone multifida var saxicola, I would want to either look into the specific differences myself or just let the process play out as normal.
Basically, sometimes and independent expert is needed, and sometimes a juror who can just weigh the credibility of existing experts is needed. So basically I think that the ‘one person one vote’ thing can actually do a better job of appropriately weighting expertise than any reasonably simple algorithm would.
On the other hand, one flaw is that when there is a maverick ID the community taxon will never differ from the display taxon; an observation can be just one vote from being RG at subspecies but nevertheless be stuck all the way back at ‘life’. One possible tweak that I think could be a huge improvement is to change just the display settings in Identify, to make IDs where there is a clear majority but not a CID actually show up in searches. Or improve search parameters to make it easier to search for observations where there is a clear majority for a specific taxon but not a 2/3+1 consensus.
It is possible to help override a wrong ID without blindly agreeing with an expert. In the example you mention, I don’t see any reason why you couldn’t provide an ID at the level of order or family if you are sure it is a sea anemone but aren’t qualified to determine the species or subspecies. This will still get the observation out of “life” and into the right general taxon, albeit not at species level. There is no requirement that you can only add a species-level ID just because the previous IDers did so.
Because in that case there is precisely zero substantive disagreement, just a typo of entering a similar sounding animal instead of the correct plant. If the obviously typo-ed ID were simply deleted, it would already be RG at subspecies. Therefore it is just wasting other IDers time to leave it in needs ID, especially at a high level. If a third expert who reviews RG observations did come along and disagree to a different subspecies, their one additional disagreeing ID will be enough to knock the CID back, exactly the same behavior as would occur if there were 2 expert subspecies IDs and no typo ID. In that case the now-unnecessary third subspecies ID can just be withdrawn to let the experts hash it out.
Like I said, I would only really consider applying this logic if the original ID was so obviously wrong it shouldn’t even be relevant in the CID calculation, and the alternate ID was at least facially plausible. The logic also isn’t relevant for taxa like birds, where there are just so many experts that the CID only needs to get to ‘bird’ for the process to kick in. Seriously, is there even one single non-meme plant observation (other than observations of the day) that has sixteen active IDs?
yes, it’s only in these sort of circumstances that I would blind agree as well
and even then I hesitate (or usually do nothing) as it nevertheless goes against community etiquette
The problem is that there are many instances where an initial maverick ID by a user who for example can’t tell the difference between a bee and a fly… can prevent an observation moving beyond order for many years…as shown in the observation linked in the other thread - https://www.inaturalist.org/observations/81586475
There simply aren’t enough sawfly experts, stilt-legged fly experts, etc to have the necessary 3 users to overcome a clearly incorrect initial ID …( in some insect families there may only be 1 or 2 people globally that can determine the species (never mind the number of those experts active on iNaturalist! )).
Why does it make a difference whether the wrong ID was due to a typo or some other reason?
If I add a species or subspecies ID without knowing why it is that species, I may be erroneously making an observation “research grade” (and thus less likely to be checked by people who actually know how to ID it). I’m not comfortable taking responsibility for that. If I add a higher ID at a level I am confident about (say, genus or family), it can be found by specialists and all it takes is one more person to make it RG. Or I do research/ask the experts who provided the previous IDs how to determine the species so that I can confirm the ID in good conscience.
I mostly ID flowering plants and bees, so I am fully aware of the scarcity of skilled IDers in many taxa. I’m also fully aware of how easy it is to end up with incorrect RG observations in these taxa because someone blindly agreed with an expert (or two!) who made a mistake.
Just because it would likely (but not necessarily)** have been RG already without the mis-entered ID doesn’t mean I should feel empowered to make IDs that I wouldn’t make under other circumstances.
Wrong IDs happen for all sorts of reasons:
incorrectly selection of a taxon with the same common or scientific name in a different kingdom or family
selection of a different item than one intended from the drop-down menu
inadvertent disagreement due to confusion about the taxonomic classification (e.g. not realizing that springtails aren’t insects, or that liverworts don’t belong to Bryophyta, or that the family Apidae doesn’t include all bees)
user blindly following the computer suggestion without any particular conviction of their own that the ID is correct
lack of consensus about what the organism of interest is (e.g. a user providing an ID for one organism in the photo when the observer is interested in something else)
mistakes such as overlooking or misinterpreting key features, or mixing up one’s geography
etc.
As frustrating as it is to try to override wrong IDs (especially multiple wrong IDs) by unresponsive users, it seems to me that making assumptions about when a conflicting ID was intentional vs. unintentional is problematic as a justification for treating certain observations differently than usual. While typos that belong to a completely different kingdom may seem “obvious” and clear-cut, there isn’t anything that makes them inherently different any other kind of mistake, many of which are less easy to distinguish from genuine disagreements about the ID of the organism in question.
If I make an ID, this should not be based on a judgment about whether I think previous IDs were entered in error, or which user I believe is more likely to be correct, but on actually being able to assess the correctness of the previous IDs.
.
.
.
** “Not necessarily” because the second person who provided a (sub)species ID could have been doing so based on their own knowledge – or they could have been confirming the previous ID to help get the observation out of “life”. Likewise, if the observation had been in the correct genus initially, some other taxon specialist might have already disagreed with the (sub)species ID and suggested a different one. The people looking at observations with kingdom disagreements are not necessarily the people with the most expertise about the taxon in question.
No
2 wrong + 1 right needs the basic 2 agree on right, plus 2 more to counter the wrong ones = 2 wrong + 4 right to reach CID.
I have painfully learnt, if the wrong are still active and willing to be encouraged then
1 wrong, the other wrong changing to right = 1 wrong + 2 right will only need one more.
Adding insult to injury is Ancestor Disagreement. If the second ID is plants 'cos it’s not that species - the algorithm says TWO wrong, I want 5 right.
PS two agree, the third doesn’t is Pre-Maverick so those can be searched for by people who want to resolve where they can.
What do you mean by ‘erroneously’? Research grade is just a check mark; it doesn’t mean a researcher can or should use the observation in a published paper about the taxon without checking it themself first. As someone who checks RG IDs myself sometimes, it is hundreds of times easier for me to find a mistaken RG observation if it is at a similar species in the right genus, rather than at a high level; I’m not about to go through all 980,000+ angiosperms stuck at class just in case one happens to be a mis-ID’d rare endemic I am interested in, but I can skim through the cover photos for all 200 RG observations of a particular similar looking species in a few minutes. Sure, some taxa don’t have enough people to review RG IDs at present, but maybe an expert with time to review a few hundred IDs, but not close to a million, will scroll through someday.
In an ideal world, absolutely. But iNaturalist observations are subject to an algorithm with such arbitrary limitations that they negate this logic.
To not add an ID to give weight to expertise when clearly the community taxon should be placed to the taxon they suggest is just limiting the granularity of the data here and shooting ourselves collectively in the foot.
This has to also be weighed up against the issues with the user interface design, which actively encourages new users to blind agree in any case! This problem causes far more issues with incorrect data points than any advanced user blind agreeing to expert determinations ever would… and makes it ludicrous to be dogmatic about not adding weight to expert IDs under certain circumstances.
I don’t know. I would never give an ID just to “gamble” the system. Maybe I hold the ID function in too much of a regard, but I would give ID if and only if I am certain of the ID, as it’s given, regardless of what other IDs exist. I don’t think this approach of giving ID with a specific aspect of the decision algorithm in mind is a good one - in particular it can be troublesome if the algorithm is changed at some point in the future.
This is a false dichotomy. There are lots of options between “leaving the observation to languish at kingdom forever” and “blindly agreeing to a subspecies ID”. (e.g., IDing to a level that will enable people with more expertise to find the observation; tagging relevant experts; researching the ID; asking for more information; etc.)
There is also a world of difference in the amount of knowledge entailed in “this is definitely a plant and not a sea anemone” versus “it is this subspecies” or even “it is definitely this family/genus”.
In the latter case, presumably one has the ability to critically assess, at least to a certain degree, whether an ID is plausible even if one was not previously familiar with the species in question. I would not consider an ID made on this basis to be “blindly agreeing”.
I’m aware of all the caveats that go with the label “research grade” (note my use of scare quotes). Nevertheless, in practice this label does have consequences for how an observation is likely to be used (i.e., whether it is shared with other databases and/or included in biodiversity records) and how likely it is to be reviewed by other users. It also feeds into things like the training of the CV algorithm and IDs made by other users who might be using such observations for reference images.
The fact that many users blindly click “agree” is not an argument for doing so ourselves; it only reinforces a problematic practice.
Adding an ID without knowledge also opens up room for mistakes, and along with this the possibility of inadvertently making it even more difficult to arrive at the correct community ID for the observation. However good my intentions might be, there is no guarantee that I won’t overlook a notification, or be absent from iNat for an extended period and thus fail to withdraw my ID when it proves incorrect. This of course applies to any ID, but I like the odds of not being wildly wrong better if I have a certain amount of competence in the taxon in question.
Sure, but it seems like you are just making a general argument that the threshold for RG should be higher; there is no particular reason why an observation that includes at least one complete nonsense/typo ID should be held to a higher standard of evidence than an otherwise identical observation that does not include any nonsense IDs.
And I’m not talking about cases like the bee vs fly example where the the original mistake is hardly ‘obvious’ for 99+% of people. In that case I’d be at least partly taking the experts word for it on whether it was even bee or a fly, so I’m certainly not going to blindly agree all the way to species. In that case I might agree with the experts to genus or family or wherever it seems like it will be easy enough for knowledgable people to find it in the future. Realistically, it would be an unusual circumstance for me to even find an observation like that because I’m not usually working on refining IDs on other peoples observations in pterygota where I struggle to consistently ID even to order.
I am not a taxon specialist. I am reasonably confident of plants on the Cape Peninsula. I have a constantly growing list (either they are newly active on iNat, or I have only just found them) of taxon specialists to @mention.
I prefer to tackle the Unknowns. Or the conflict / disagreement IDs. Checking each day for
My Mavericks (usually already cleared by my other URLs)
Pre-Mavericks for the Cape Peninsula
then Western Cape
recently included the rest of South Africa (since they have been ignored)
Plant conflicts in Africa
Kingdom disagreement in Africa
Skimming thru Not Wild international Pre-Mavericks (cultivated South African plants?)
Those 7 don’t take me long each day. And it is rewarding to see notifications roll in, as taxon specialists find them.
I never auto-click blind agree. I weigh up each obs and respond in many different ways depending on each situation. The algorithm won’t change to suit me, so I am forced to find a way to work around existing problems.