Does anyone else get bothered by how many observations are marked as "unknown species"?

I have a different definition of unknown knowns, keeping the subject constant: Things that I don’t know that I know. (Or that you don’t know you know, &c.)

For instance, think about reading a novel written by someone who lives in a very different social or technological world than you. There’s a massive amount of information that the author assumes the reader will have. The full extent of that information only becomes apparent when it is absent. As the confused reader in this example, many of the author’s unknown knowns are experienced by you as known unknowns.

2 Likes

Yeah, I often considered the unknown knowns to be those things buried in my brain (maybe in my reptile brain) that I’m unaware of at a conscious level but are known to me at some deeper level. But that’s probably way off topic.

Then, it can be interesting down the line to see which of those formerly Unknowns move right along from a very-high-level ID to genus or even species. Sometimes, I am so tickled when I get notifications for something ID’d at very high level that has already been reviewed by others with real expertise. After languishing in Unknowns for months or more, suddenly an observation has a relatable ID - amazing!

2 Likes

I think we may have seen an example of this in the recently closed thread. the last post before it closed said

I was puzzled by this. I went ahead and bolded the “unknown known” – that is, instead of simply withdrawing the species level ID, I am confused as to why they didn’t revise it to a broader ID of Genus Amaranthus or Family Amaranthaceae (depending on which taxon they meant by “amaranth”). Then the observation would have only gone back to that level instead of to Unknown. But this may be a case of being unaware at a conscious level of something that is known to them at a deeper level.

And no, I don’t think that it is way off topic. It’s a likely explanation for things being marked as Unknown.

Withdrawing the ID is one click. :-) Entering the broader ID is more robust to a variety of future possibilities, but this may not be obvious at the time. Absent a reason to choose one option over the other, minimizing effort is reasonable enough.

There’s something similar going on, I think, with some previous discussions in which people were annoyed by others entering coarse IDs—something like, “Anyone can see it’s a rabbit, but why would you identify it as ‘rabbit’?” Something known but, I guess, considered so trivially obvious that putting it in the ID field was objectionable. People are strange.

3 Likes

This is not a case of an observation being entered as unknown and left as such by an observer who never returns to it. You are taking things out of context again.

Once I discovered that my observation had become “unknown” on account of the other user deleting their account, I did in fact reenter an ID. (Once again, please note that it was only unknown because I did not receive any notification that the other ID was now gone. I had no reason to expect that the ID would suddenly disappear. Anyone who had looked at the observation during this time would have seen my withdrawn ID and likely speculated that something odd was going on.)

Why didn’t I initially enter a new, broader ID in response to the disagreeing ID instead of just withdrawing mine? Frankly, I saw no need. The observation had an ID provided by the other user, which I suspected was correct but was not confident confirming. (Before you criticize me again for not following up and learning how to distinguish the two species: I decided that at present I was simply not practised enough to see the relevant distinctions for this genus and that my time would be better spent trying to master some other, slightly less intimidating taxon. So I left the observation with the other user’s ID, to be confirmed by others or to return to myself at some later date.)

That is what matters to me – that the observation is labelled, ideally as accurately as possible. This was the case. I felt no need to subsequently prove that I at least know the genus by putting this as an ID. Again, anyone who looks at an observation and sees the history of IDs and withdrawn IDs can reconstruct the process that went on (provided that part of this history doesn’t simply disappear without warning).

I have a number of older observations where I originally entered a relatively broad ID which I could now correctly identify to species level. In the meantime, they have reached “research grade”, sometimes with multiple confirming IDs. I suppose I could go back and add my own species ID now, but again, I don’t see any reason. I was where I was in the past; now I know more but I prefer to apply this acquired knowledge to future observations rather than adding another, unneeded agree to an existing one.

3 Likes

All I know is I’m pretty sure I know less now about all things that are knowable than I thought I knew decades ago. Knowing one’s own ignorance is a learning process.

7 Likes

Just to ensure that complaints about off-topicness have some justification, I ran across a nice real-world example of unknown knowns. I bought a laser rangefinder. The battery compartment has a little lid that unscrews, which the designers provided with an unlabelled directional arrow.

Does the arrow point in the direction to open the battery compartment, or the direction to close the battery compartment? The designers surely knew, but it did not occur to them that this was a thing they knew that other people might not know.

(It points in the direction to close the compartment, as it happens. I guess they’re not worried about how you’ll open it, but think you might forget which way it goes when you close it.)

2 Likes

I agree it is important. I do remove (disable) my wrong IDs.

1 Like

This remark sounds very important and we might ask for a new feature: the top suggestion (displayed as “We’re pretty sure this is…”) should be a taxon covering almost all the species suggestions displayed. It would often be a high rank taxon, Order or Family, a name that more people would know, like butterflies.

I am not actually asking for removing the present top suggestion, but maybe adding one more on the top of it. The taxon suggested would also (partially?) satisfy the persons asking for automatic identifications, yet the ID would not be put, just strongly suggested:

And the icing on the cake would be to make this top ID suggestion searchable… see other discussions about a similar feature, for instance here.

In this example where species of different Superfamilies are displayed, yet a Genus is proposed as the top suggestion (which at first sight seems contradictory, isn’t it ?) (the reason is that the species suggested have extremely different confidence scores, but these scores are not displayed):

https://www.inaturalist.org/observations/138013069

BTW, the computer vision has been much improved and is trained on a larger and larger set of species, so does it still make sense to display as suggestions several taxa that have a confidence score near zero?

3 Likes

We have a global issue with “complex feature requests”.

We have been discussing about interesting suggestions about the “unknowns” for years, and these suggestions remain either disconnected from each other (and supported by too few people) or conflicting with each other (alternative responses to the same question), so that no actual feature request emerges, for which we would vote, in order to get it realized.

The system is not designed to promote suggestions within a discussion thread (except this thread). It is designed to work with votes for a feature request. So, we need a “feature request” containing a complete and consistent response to several needs about the unknowns and about identifying. We should collect the use cases and think about them as a whole, and elaborate a solution. We lack a shared vision. Either we build it, or we won’t get anything realized.

2 Likes

I battle each time I ID a Psoralea.
The default suggestions leap confidently straight to (wrong) species, including some new species which haven’t even been formally described yet.
Try getting iNat to offer a single click for
Genus Psoralea

iNat won, Diana still fighting back. (And the taxonomists have joined the battle now …

1 Like

Does anyone know what percentage of observations are uploaded as Unknowns? I’m assuming it’s probably only 1 or 2 percent, but I’m curious and I don’t know how to find out.

Can’t answer - but I think it has come up before in the forum.
@tiwane or @pisum can probably tell you.

Recent observations:

https://api.inaturalist.org/v1/observations?id_above=142000000
2742773 observations
100 %

https://api.inaturalist.org/v1/observations?id_above=142000000&identified=false
92190 observations not identified
3.36 %

https://api.inaturalist.org/v1/observations?id_above=142000000&identified=true
2650557 observations identified
96.64 %


Over a shorter period, same result:

https://api.inaturalist.org/v1/observations?id_above=143000000
1770788 observations
100 %

https://api.inaturalist.org/v1/observations?id_above=143000000&identified=false
61028 observations not identified
3.45 %

https://api.inaturalist.org/v1/observations?id_above=143000000&identified=true
1709770 observations identified
96.55 %


This comparison suggests that "unknown" observations get identified very slowly.
To date, the total is:

https://api.inaturalist.org/v1/observations
138831280 observations
100 %

https://api.inaturalist.org/v1/observations?identified=false
3474064 observations not identified
2.50 %

https://api.inaturalist.org/v1/observations?identified=true
135357199 observations identified
97.50 %

3 Likes

Does the 2.5% that are identified=false include Casual observations?

How many of the ‘identified’ have a very broad ID like Plantae?

1 Like

identified=false and quality_grade=casual are independent filters, I think (technically, overlapping filters with different names, identified and quality_grade, wouldn’t make sense).

Let’s do it for all ranks:

https://api.inaturalist.org/v1/observations?identified=true
135387563 observations identified
100.00 %

https://api.inaturalist.org/v1/observations?identified=true&rank=stateofmatter
92446 observations identified at rank Stateofmatter
0.0683 %

https://api.inaturalist.org/v1/observations?identified=true&rank=kingdom
1561815 observations identified at rank Kingdom
1.1536 %

https://api.inaturalist.org/v1/observations?identified=true&rank=phylum
537320 observations identified at rank Phylum
0.3969 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subphylum
549127 observations identified at rank Subphylum
0.4056 %

https://api.inaturalist.org/v1/observations?identified=true&rank=superclass
92 observations identified at rank Superclass
0.0001 %

https://api.inaturalist.org/v1/observations?identified=true&rank=class
1946605 observations identified at rank Class
1.4378 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subclass
147088 observations identified at rank Subclass
0.1086 %

https://api.inaturalist.org/v1/observations?identified=true&rank=infraclass
13931 observations identified at rank Infraclass
0.0103 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subterclass
3715 observations identified at rank Subterclass
0.0027 %

https://api.inaturalist.org/v1/observations?identified=true&rank=superorder
34174 observations identified at rank Superorder
0.0252 %

https://api.inaturalist.org/v1/observations?identified=true&rank=order
1998206 observations identified at rank Order
1.4759 %

https://api.inaturalist.org/v1/observations?identified=true&rank=suborder
340868 observations identified at rank Suborder
0.2518 %

https://api.inaturalist.org/v1/observations?identified=true&rank=infraorder
200858 observations identified at rank Infraorder
0.1484 %

https://api.inaturalist.org/v1/observations?identified=true&rank=parvorder
2309 observations identified at rank Parvorder
0.0017 %

https://api.inaturalist.org/v1/observations?identified=true&rank=zoosection
14630 observations identified at rank Zoosection
0.0108 %

https://api.inaturalist.org/v1/observations?identified=true&rank=zoosubsection
48279 observations identified at rank Zoosubsection
0.0357 %

https://api.inaturalist.org/v1/observations?identified=true&rank=superfamily
515862 observations identified at rank Superfamily
0.3810 %

https://api.inaturalist.org/v1/observations?identified=true&rank=epifamily
58778 observations identified at rank Epifamily
0.0434 %

https://api.inaturalist.org/v1/observations?identified=true&rank=family
4202948 observations identified at rank Family
3.1044 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subfamily
1500456 observations identified at rank Subfamily
1.1083 %

https://api.inaturalist.org/v1/observations?identified=true&rank=supertribe
1378 observations identified at rank Supertribe
0.0010 %

https://api.inaturalist.org/v1/observations?identified=true&rank=tribe
1024656 observations identified at rank Tribe
0.7568 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subtribe
200645 observations identified at rank Subtribe
0.1482 %

https://api.inaturalist.org/v1/observations?identified=true&rank=genus
16935667 observations identified at rank Genus
12.509 %

https://api.inaturalist.org/v1/observations?identified=true&rank=genushybrid
1439 observations identified at rank Genushybrid
0.0011 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subgenus
432888 observations identified at rank Subgenus
0.3197 %

https://api.inaturalist.org/v1/observations?identified=true&rank=section
230939 observations identified at rank Section
0.1706 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subsection
17706 observations identified at rank Subsection
0.0131 %

https://api.inaturalist.org/v1/observations?identified=true&rank=complex
319106 observations identified at rank Complex
0.2357 %

https://api.inaturalist.org/v1/observations?identified=true&rank=species
99251656 observations identified at rank Species
73.309 %

https://api.inaturalist.org/v1/observations?identified=true&rank=hybrid
321007 observations identified at rank Hybrid
0.2371 %

https://api.inaturalist.org/v1/observations?identified=true&rank=subspecies
2351847 observations identified at rank Subspecies
1.7371 %

https://api.inaturalist.org/v1/observations?identified=true&rank=variety
501751 observations identified at rank Variety
0.3706 %

https://api.inaturalist.org/v1/observations?identified=true&rank=form
24285 observations identified at rank Form
0.0179 %

https://api.inaturalist.org/v1/observations?identified=true&rank=infrahybrid
3068 observations identified at rank Infrahybrid
0.0023 %

4 Likes

So drifting towards 5% or 1 in 20 are ‘unknown’ if it comes to the crunch. Not nearly as bad as it feels if you are IDing thru that 5%.

4 Likes