Options for the best way to handle non-established obs (e.g. escaped/released pets)

[loarie]

What do we show by default on Explore?

While iNaturalist has many uses, one of the core audiences of the site is the natural history and biogeography communities who are interested in the distribution of established populations.

This is why, by default on the iNaturalist Explore page, we filter by observations that have all their core attributes (media, location and date not missing or flagged as inaccurate) and also are wild (e.g. not cultivated plants or pets).

The graph below shows the flow of observations posted to iNaturalist into each of the 3 bins: Casual, Needs ID, and Research. This subset of observations that are missing attributes or are captive/cultivated end up in the Casual bin. Everything else (Needs ID + Research) ends up in this subset that we filter on the Explore page by default. Observations with a community taxon at the species level are Research, otherwise they are Needs ID.

We can thus achieve our goal of filtering observations that are not missing attributes and are wild on the iNaturalist Explore page by default by filtering for NeedsID+Research (but not Casual) observations.

Not Established Obs such as Escaped and Released Pets

For the same reason we don’t want to show captive/cultivated observations by default on Explore, we also don’t want to show observations of released or escaped pets or individuals that otherwise don’t represent established populations. We feel that narrowing one’s focus on established observations is useful to:

1. Highlight mis-ID’d taxa
2. Highlight discoveries of new established populations
3. Browse iNaturalist like one would browse a field guide (which typically include established native and introduced species, but not escaped pets)

And while the number of escaped observations is tiny, often a significant number of species in the species tab are driven by these observations. Non-established observations include things like escaped and released pets, hitchhikers (e.g. the frog in the Home Depot plant). Obviously, there are gray areas with regards to vagrants and species established in greenhouses etc., that we’d need to come up with norms for. We’d also need to come up with norms for plants which are perhaps less clearcut than animals? And sometimes establishment is difficult to know for sure (e.g. long-lived turtles in ponds). You can see examples of observations currently being tracked as non-established with this observation field.

Related to this, there is ongoing ambiguity within the iNaturalist community about the meaning of the ‘wild?’ Data Quality flag. Some people interpret it as captive/cultivated at the time of observation (e.g. ‘is this snake in a terrarium?’). Others interpret it to mean ‘member of an established population’ (e.g. ‘is this snake on the lawn an obvious escape from my neighbor’s terrarium?’). We’d like to bring more clarity to how the ‘wild?’ flag should be used.

We’ve discussed several options for how best to track non-established observations from the existing ‘wild?’ flag, the existing ‘Escapee/Non-Established’ observation field, to a new Annotation. Our preference would be to create a new data quality flag similar to but separate from ‘wild?’ - e.g. ‘established?’

This would avoid concerns about not being able to easily filter escapees separate from captive/cultivated observations (if we were to use the broader interpretation of the existing ‘wild?’ field) as well as concerns about involving observation field or annotations in default Explore searches (if we were to use the ‘Escapee/Non-Established’ observation field or a new annotation).

Other advantages of this approach is that it would be very quick to implement. We’d simply:

1. Add a new ‘established?’ data quality flag
2. Clarify that ‘wild?’ refers to captive/cultivated does not include released/escaped pets

We could quickly make the new ‘established’ data quality flag parameter available in a url search to filter for unestablished observations, it may take us a bit longer to add a ‘established’ button to the Explore page filters.

Disadvantages are that we suspect some will take issue with these escapee observations being casual for the same reasons that some have taken issue with captive/cultivated observations being casual. The main argument is that captive/cultivated observations can still often be identified and are applicable to certain research projects so the name ‘casual’ isn’t appropriate.

More ambitious alternative

Another more ambitious/disruptive alternative would be to bundle this work with additional work to remove ‘wild? No’ from casual as in the graph below.

This would mean that captive/cultivated observations would be in Needs ID and Research. But we would add an additional set of search filters to Explore that default to excluding Casual, Captive/Cultivated, and Non-established observations by default.

The advantages would be that captive observations can become ‘research grade’ which might encourage people to identify them. You also would able to search for captive observations with out a species level Community Taxon (by searching for what will become Needs ID, captive/cultivated observations) which you can’t do now.

The disadvantages here are that this won’t do much to extend what people can already search for, it mainly just alter the philosophy of what casual should or should not include which might displease as many people as are pleased by the change. It also adds a new dimension complexity to accommodate a tiny fraction of arguably less relevant captive observations. Lastly, it will be a much more ambitious (longer time frame) project to implement requiring:

1. Add a new ‘established?’ flag (as above - but not part of the Research Grade calculation)
2. Update Research grade (RG) calculation to also remove ‘wild?’ from the calculation
3. coordinate RG update with changes to/impacts on many parts of the site including:

• check lists
• atlases
• the computer vision model data export script
• every project that has an existing RG requirement
• every partner consuming our data that may be assuming RG means what it currently means.
• relevant documentation that refers to the current definition or RG

4. On the observation page make clear that ‘wild?’ And ‘established?’ aren’t part of the RG calculation
5. Add new search filters for Captive, Non-Established, and Established for Explore and Identify.
6. Set the default search filters on explore to be Needs ID + Research (not Casual) and Established (not Captive or Non-established)

We’re curious what you think about these options given these tradeoffs.

[Star3]

@loarie

we also don’t want to show observations of released or escaped pets or individuals that otherwise don’t represent established populations.

But aren’t we evaluating each organism on it’s own? If there is a single escaped/released pet, it’s still effecting the ecosystem (especially if it is a prolific predator) even if it isn’t part of an established population. Also, it could be the progenitor of a future established population, and who is going to go back and retroactively tag these early non-established escaped/releases as established in a few years?

[Star3]

@loarie

This would mean that captive/cultivated observations would be in Needs ID and Research.

Because “Needs ID” really means “Needs ID to achieve Research Grade”?
Could we have a way for something to be “Needs ID” just because it Needs an ID (or a finer ID), without it ever being eligible for RG?
I’d like to see captives (appropriately tagged as such) in “Needs ID” because they need a finer ID (or any ID) without ever getting to be RG. That way people can filter captives out still, and we have the advantage of encouraging new users, because those who don’t filter out may still give them an ID since it shows up in the “Needs ID” pool.

[addendum: link to relevant feature request topic]

[loarie]

Could we have a way for something to be “Needs ID” just because it Needs an ID (or a finer ID), without it ever being eligible for RG?

Yes, thats another easy option. We could add a search filter for ‘has community taxon’ then if you searched for captive + has_photo + has_community_taxon=no + rank = ‘species’ you would see all the captive obs at species level that still need a community ID to mostly simulate ‘needs ID’ . (you can already search for captive + has_photo + lrank= genus [ie coarser than species] which also ‘need IDs’)

If there is a single escaped/released pet, it’s still effecting the ecosystem

Yes, and under all options being discussed you’ll still be able to post, ID, and search for observations of escapee/released pets for these reasons. We anticipate that if a species becomes established, people will change the way they flag observations of that species/place based on that new information. As mentioned in the doc, even though the number of escaped/released pets is tiny they can dominate the ‘species list views’ for a place (ie the species tab on explore searches) for example scroll down on that link and you’ll see the tail is dominated by released/escaped pets that don’t represent established populations mixed in with species from mis-ID’d obs. If we were to clarify that the norm for ‘wild?’ is that it does not cover escaped/released pets this issue will become even more exacerbated and makes it very hard to spot outliers in the form of mis-ID’s and rare established species. Our opinion is that by default the species tab on explore should represent this ‘field guide’ scope of a place to help keep iNaturalist’s culture focused on natural history - but anyone who wants to include escaped/released pets will be able to change the defaults to display them

[Star3]

@loarie

We anticipate that if a species becomes established, people will change the way they flag observations of that species/place based on that new information.

Which affects new observations, not ones already tagged, who might have been part of a population that was not yet recognized at the time they were tagged.

Unless someone decides to periodically review places with established populations and look for old observations that had been tagged as escapees and evalute whether to switch those tags or not on each observation.

@loarie

As mentioned in the doc, even though the number of escaped/released pets is tiny they can dominate the ‘species list views’ for a place (ie the species tab on explore searches) for example scroll down on that link and you’ll see the tail is dominated by released/escaped pets that don’t represent established populations mixed in with species from mis-ID’d obs.

@loarie

Our opinion is that by default the species tab on explore should represent this ‘field guide’ scope of a place

If they are that dominant, aren’t they very likely to be encountered and should be included when using explore as an informal ‘field guide’?

[loarie]

If they are that dominant, aren’t they very likely to be encountered and should be included when using explore as an informal ‘field guide’?

no - they represent a tiny tiny subset of the observations so they don’t have a noticeable impact on the observations tab on the explore page - but because the diversity is more or less unbounded (in theory any animal species can be in captivity and can briefly escape/be dumped and be observed) they have a very noticeable impact on species tab of the explore page. For example in my CA reptiles example escaped/released pets represent about 0.02% of observations but about 20% of species

[arboretum_amy]

@loarie

This would mean that captive/cultivated observations would be in Needs ID and Research. But we would add an additional set of search filters to Explore that default to excluding Casual, Captive/Cultivated, and Non-established observations by default.

Not going to lie, I’m shocked you have this on the table at all given the very forceful negative reactions to captive/cultivated observations I have read in past forum discussions. I’m all for it personally, but if I were you I’d expect backlash.

@loarie

We could add a search filter for ‘has community taxon’ then if you searched for captive + has_photo + has_community_taxon=no

I find this to be the biggest need. Yes, philosophically speaking I’d prefer the the other option of removing the casual label from captive observations, because to my mind it’s offensive to place them in the same “bin” as observations missing their photo/date/location/etc. (Disclaimer of bias: I’m a professional horticulturist.) But functionally speaking, being able to search for captive plants which don’t already have good IDs would be the most relevant to my site use habits.

[jnstuart]

In various faunal checklists for a particular region, I’ve seen the term “waif” used for those documented occurrences of an animal species that has shown up maybe once or twice in an area and for which there is no evidence that it has become established. Sometimes they are known to be popular pets. However, since there is the chance the species could become established, the record is of interest. Don’t know if it would be easier to create a new category of “waif” (or whatever other term might be more readily understood) for these cases of “introduced but not established.”

Of course, not all “waifs” have the same potential to become established. We had a Serval (presumably an illegal pet) loose near Santa Fe, NM a year or so ago … the chance of that cat becoming established is virtually nil. But our state’s population of Mediterranean Geckos probably started out as “waifs” (perhaps pets, but probably hitchhikers) and are now well-established in several areas and probably in process of establishing in others. Florida has more than its share of escaped/liberated herpetofauna that started out as pets and are either established or potentially on their way to becoming so.

[arboretum_amy]

I also suggest changing the title of this thread, because even though pets (animals) are the focus for you, a lot of plant people are going to want to read this.

[ajamico]

@loarie

coordinate RG update with changes to/impacts on many parts of the site including:

@loarie

• every project that has an existing RG requirement
• every partner consuming our data that may be assuming RG means what it currently means.
• relevant documentation that refers to the current definition or RG

I’m not really sure this would be helpful and is probably more ambitious than necessary, but what if there was a fourth bin, Research (Non-Established) (“RNE”)? This might make it easier for project creators and data partners to decide whether they want to include this subset of research-grade observations. It would be a little difficult to determine when a species has truly become “established”, and whether that means that any RNE observations of that species in that area would change to normal RG, but it might be a little more useful than just lumping them into RG.

I’m also thinking about the example of American Flamingos in Florida that were native, were killed, and then were “reintroduced” accidentally by escaped pets (along with some flying in from Cuba). Would observations of these be considered native? If an observation is of a descendant of escaped pets, is the offspring also considered an escapee? It does raise a lot of questions and ambiguity surrounding this, but I guess it’s not more than the “Wild?” tag already is.

[arboretum_amy]

@jnstuart

I’ve seen the term “waif” used for those documented occurrences of an animal species that has shown up maybe once or twice in an area and for which there is no evidence that it has become established.

We use that word for plants too. Specimens which arrived with dumped soil, or washed down a canyon from someone’s hilltop home, etc.

[jnstuart]

The zoologists probably borrowed the term from the botanists.

[malisaspring]

I’d think having a DQA question for “Is it established?” might confuse some people and they would then accidentally make things casual grade. I could see some people interpreting that as “has it established well?” - like a good root system on a tree versus a stunted, root bound tree. Or is it established socially? (ie - good social standing within a group for social organisms or is it an outcast). But this might be more of a semantics issue, so hopefully someone might be able to come up with slightly better wording to make it clear (just like how we have issues with people confusing the “is it wild?” DQA)

I’m trying to think how I would answer the way I think you want your question interpreted for many of my weird observations and my answer would be “I don’t know”. I wouldn’t feel I would be able to say for sure for a lot of groups without really digging into the literature. And even my few observations where I am pretty sure the organisms do not have a viable population, I think they might be able to in a few more years with a few more reintroductions given our rate of warming.

I do like your ambitious alternative suggestion, as I know a lot of people who submit cultivated plants would like more feedback. Though I don’t know that changing how things are categorized would bring on that many more identifications considering the huge backlog for plants.

[Star3]

@loarie

(you can already search for captive + has_photo + lrank= genus [ie coarser than species] which also ‘need IDs’)

or + sounds
Is there a way to search for with (sound OR audio) without it expecting both to be true?

@loarie

no - they represent a tiny tiny subset of the observations so they don’t have a noticeable impact on the observations tab on the explore page - but because the diversity is more or less unbounded (in theory any animal species can be in captivity and can briefly escape/be dumped and be observed) they have a very noticeable impact on species tab of the explore page

Ok, so we’re looking from very different perspectives.
I don’t usually use the explore pages on places, and for identifying observations I look at the organism’s “wildness” on it’s own.

[cthawley]

I think trying to do this is pretty problematic. It’s really hard to determine what is “established” and what isn’t from iNat data. I’ve found multiple new populations of invasive species (animals) from iNat data that would have been flagged as “non-established” initially.

These are also some of the most useful observations on iNat (at least in terms of invasion biology). Having them get booted from the general workflow of the site and from export to GBIF would be a big loss (assuming that I am understanding what is being proposed correctly). Researchers with expertise can always evaluate for themselves whether the data are usable or not (or are established enough or not), but this change would seem to make the data much less accessible.

I also agree that the title of the post is a bit misleading. It’s way broader than escaped or released pets and the proposed change is pretty far-reaching in my view.

[Star3]

@cthawley

I’ve found multiple new populations of invasive species (animals) from iNat data that would have been flagged as “non-established” initially.

I think that’s where Scott was proposing people will stop selecting “non-established” once a population becomes so:

@loarie

We anticipate that if a species becomes established, people will change the way they flag observations of that species/place based on that new information.

I agree with you, though, because it could be that observations that were tagged “non-established” originally were in fact already part of a population that just wasn’t fully known yet.

[cthawley]

I agree with @Star3 I think there would be a lot of confusion about when to start calling something established vs not, and then you’d have the issue of some older records being listed as non-established while others are established.

I don’t think folks would periodically review all the places to see whether things had become established or not and then go back to search for previous records that had now been shown to be established.

Also, as mentioned elsewhere, I think it is really difficult/problematic to tell whether something is established or not from an iNat observation alone. For instance, iNat just celebrated this observation of a plant found 100’s of km outside its known range. It was an observation of the week and featured on the celebratory 50 million observations post. But under this system would this observation have been nailed as “non-established” and relegated to the backwaters where it might not have been noticed?

Lastly, I think seeing all the weird species that show up is actually a feature of iNat in some ways. If I want a field guide, I’ll get one. iNat shows the other organisms that people are finding on the ground which can be very valuable if you want to know what you have in a given place. This is the type of info that isn’t available in other sources, it is useful, and I wouldn’t want to lose it.

[raymie]

I am probably the user most actively looking for observations of escapees at this point. I have been voting every escapee observation I can find as wild. I have some input here.

First of all, the “escapee/non-established” field is NOT a good way to find observations of these. Too many people have applied the field to populations that are established, like Red-crowned Parrots in Los Angeles, Green Iguanas in Singapore, ect. My project “Escapees from Captivity” (https://www.inaturalist.org/projects/escapees-from-captivity) is a much better collection of these observations. My project does also not include hitchhikers (such as Brown Anoles hitchhiking in house plants) although I do believe these should be marked as wild as well (at least for now, until this new system, whatever it is, is implemented).

I think the option that would be easiest is for escapees to be considered RG with a vote in the DQA saying something like “Is this established?” which would not contribute to whether the observations is RG or not. When viewing observations through Identify or Explore these observations would show by default but could be filtered out if one so chooses. I think this should only be used on observations of animals though, as garden escapees are a different matter altogether.

However, I think this “Not Established” option would have to also have to apply to natural vagrants as well. I simply see no way to avoid it.

Another issue is I fear people might not do their research before voting “Not Established”. If I see an observation that looks to be an escapee, I always check around online to make sure it is an escapee and not part of an established population, but I highly doubt very many people would do this before voting “Not Established”. Even eBird reviewers in many parts of North America tend not to accept records of species that are not officially “countable” by ABA standards, even when said populations have been self-sustaining for decades. And seeing that I have seen iNat observations of animals that have been established for more than a century marked as “Not Wild” I have little hope that things would be any different on this site

[cthawley]

Right. You could get 10 observations in an area uploaded every few weeks. But if the previous observations keep disappearing from the map of potential nearby sightings, how would we ever know that they are established? The next one will always seem like it is “non-established” all by its lonesome…

Unless the same person is doing all the iding of a given species in a given place, they wouldn’t know?

[loarie]

It’s really hard to determine what is “established” and what isn’t from iNat data.

True, there would be discussions - like there currently are around the wild? flag and the Escapee/Non-established observation field on a case by case basis. I’d expect for well known clades/places the expectation would be not-established unless good evidence otherwise. Whereas for less well/known clades/places the expectation would be established unless good evidence otherwise.

These are also some of the most useful observations on iNat (at least in terms of invasion biology).

You would still be able to find these observations by searching for non-established obs (much like you can currently use the observation field https://www.inaturalist.org/observation_fields/103).

Having them get booted from the general workflow of the site and from export to GBIF would be a big loss (assuming that I am understanding what is being proposed correctly).

I’d be surprised if GBIF wants released / escaped pets given all the work they do using spatial filters to exclude outliers (like bearded dragons in North America). But regardless - GBIF can harvest these if they want them regardless of how we label them

I also agree that the title of the post is a bit misleading. It’s way broader than escaped or released pets and the proposed change is pretty far-reaching in my view.

I changed the title after arboretum_amy’s suggestion