Add geographical boundaries for accuracy experiments (Experiment 0.6)

Feature Requests
1

Description of need:
As a part of Experiment 0.6 I was asked to ID plants and animals across Australia, most are completely outside my area of knowledge as I am Tasmanian based

Feature request details:
Based on a users identifying history would it be possible to only ask users participating in the accuracy experiments to identify observations in a particular state or territory. Over 90% of my ID’s are in Tasmania and primarily of plants however as part of the experiment I was asked to identify plants in the NT, QLD, NSW, VIC and SA in addition to TAS within Australia and naturally I am not able to provide a useful ID for the Experiment because of this.

2

I believe the restriction to country is newish. But yes, they could also restrict further.

3

I had observations in 4 different countries in my sample (Australia, Bolivia, Brazil, USA), which I didn’t mind. I don’t know how they decide which observations to give to which person, but I personally wouldn’t want to be restricted to just one area.

1 Like
4

i get how being shown a bunch of things outside your comfort area might feel a bit weird, but is there any reason you can’t just manually filter these out? (if i recall correctly the accuracy experiments opens a version of the identify page with normal location filters available)

i can easily imagine cases (especially for more rarely observed taxa) where an identifier is able to make an identification but hasn’t happened to ID that taxon in that particular state ever, and it seems really good to at least give those IDers a chance to make an identification, and if iNat wants to trust users to self-determine which obs they are qualified to ID that seems ok to me.

feel like i could be easily missing something but is there anything this is meant to solve aside from the (i think small) inconvenience of having to apply a place filter?

5

For the experiment you are asked to add an ID to every observation in your sample - so filtering by place wouldn’t fit with that request.

4 Likes
6

oh I see, that seems odd in and of itself. why not just instruct people to ID what theyre comfortable with?

7

Same here. I too live in the island state of Tasmania. To the accuracy experiment, I must look like an expert identifier to species level of several genera. But that’s because we only have a few of them:

The accuracy experiment looks like it needs better awareness of geographic boundaries, and how a genus can be much harder to identify in one place than another. Without that awareness, it is going to make serious errors about who the experts in a given genus are. For the genera I have named here, expert identifiers in Victoria will be much less confident than I am (because they have more species to distinguish between) but demonstrate greater skill than I have. This isn’t just me - every observation I’ve been given in the accuracy experiment has a half-dozen Tasmanians acknowledging that genus is the best we can do.

3 Likes
8

I apologize; I couldn’t find the right thread for this comment, so I’m writing it here.

I participated in Observation Accuracy Experiment v0.5, so I think I meet the validator candidate criteria for Observation Accuracy Experiment v0.6?

But I didn’t receive an invitation. No email in my inbox, and no DM from “admin” on iNaturalist.

9

it might just be that the pool of 10k random observations had taxa you met the threshold for in 0.5 but didnt in 0.6

2 Likes
10

Because that is how they set up the experiment.

I got 2 trees in Costa Rica … I can dicot.

to @AdamWargon the selection is random, in one round might be out next time (especially in USA where the identifier pool is large)

1 Like
11

I prefer not to have those geographical boundaries placed. Many people ID things from areas outside where they live, and even more so when it comes to experts in their fields.

If you’re not comfortable adding an ID then either don’t do it, or keep it to only what you’re confident in. Just like any other observation on the site.

2 Likes
12

I was going to suggest something similar, but with a slight adjustment that might satisfy those taxonomic specialists who ID worldwide. How about giving participants a sample from the smallest “standard” place that includes 90% (or maybe 80% or 95%) of their IDs. My sample this time included all of the USA, but my expertise is mainly in Oregon and California, so I had to go with Family or Genus level on much of my sample, which shouldn’t bias the results but could lead to much higher uncertainty in the “percent correct” statistics.

13

iNat has no reliable way of knowing where people live. It has some information about where people frequently observe and where they ID; neither of these are necessarily the same as where they live. But I don’t think this request was asking to restrict selection criteria to region of residence.

In some iterations of the experiment, validators were selected based on whether they had provided IDs for the taxon in question within a certain radius of the observation location, as a reasonable proxy for whether a person is likely to have relevant expertise to ID in that area. I believe if not enough validators were found who met the criteria, the search was broadened to users who had IDs for that taxon in more distant regions.

Validator selection is only one of the issues that has affected these experiments from the beginning. Another challenge for validator selection is related to the inclusion of observations with high-level IDs in the experiments, because people who provide high-level IDs are often generalists and might not be able to provide insight into whether the ID can be refined or not.

1 Like
14

I wasn’t saying that iNat did. That portion was in direct response to the initial post as they stated that they are based in a particular location and do most of thier IDs in that location.

1 Like
15

But the request does not propose limiting selection based on residence. It merely suggests refining the selection criteria to help ensure that validators have relevant expertise in the region where they are asked to validate observations. In the original post, this correlates with the place of residence, but the request does not make any specific suggestion about how the geographical boundaries are to be determined, nor does it assume that for all users the place of residence will correspond with geographical expertise.

16

A good example of this is the user juncuspike who is an expert on Australian Juncus species who lives in the Netherlands. It would make sense for this user to be provided with observations in Australia where they frequently ID these plants