Automatic iNat suggestion for "unknown" observations that reach a certain age

That is what I do for my Unknowns, and the finer IDs roll in promptly from people who have filters set to Lepidoptera.

2 Likes

Beware that this bug might have affected the selection of observations that I pushed recently to the “unknown” observations projects mentioned above:
https://forum.inaturalist.org/t/random-id-suggestions/37978
https://forum.inaturalist.org/t/more-recently-the-ai-with-inaturalist-seems-to-be-failing/38010

Beside losing data, providing wrong data is the worst thing a server can do.

I will not check if I really got wrong data in cache, I just discard everything I got from the 12th december, by restoring a backup I made, by chance (in principle, I trust the server), on 11th december 2022. I will not do anything for checking observations already pushed to the projects. Just be cautious before concluding anything if you find some irrelevant observations in these projects.

2 Likes

Journal post:
https://www.inaturalist.org/journal/jeanphilippeb/73398-draft-for-creating-projects-for-unknown-observations


Entry point for the new Fabaceae project and sub-projects:
https://www.inaturalist.org/projects/unknown-fabaceae

Sub-projects already populated with some observations:
https://www.inaturalist.org/projects/unknown-fabeae
https://www.inaturalist.org/projects/unknown-trifolieae

2 Likes

New subprojects for Lepidoptera.

The project Lepidoptera has been deleted and recreated, in order to be populated again, but only with the observations that will not go to the new subprojects.

In other words, this action is equivalent to a reset of the statistics and to the removal from the project of all observations that have been identified. This will result in a better sort of all observations still to identify. It happens only because the first Lepidoptera project was created as an experiment, without regard to the future plan. This should not happen again.

Links to project and subprojects:

Order Lepidoptera
     Superfamily Bombycoidea
     Superfamily Noctuoidea
     Superfamily Papilionoidea
               Subfamily Papilioninae
          Family Nymphalidae
               Subfamily Nymphalinae

In the description of the project Lepidoptera, there is an “umbrella” link for identifying all observations at a time (in project and all sub-projects recursively). No need to browse the subprojects one by one.

2 Likes

Main project has more observations of plants than butterflies (in verifiable).

Not surprising that observations that are “difficult” for the computer vision go to a higher rank project (as a consequence of more scattered c.v. suggestions, I guess). But I don’t see that many, what is the URL that shows more observations of plants than lepidoptera?

The projects were still being populated, maybe the ratio was different a few hours ago.

It was when there were about 10 pages less, the most current one is filled with moths.
They should stay in the project, right? So, I ided about two pages of them before.

Yes, if you have just identified them (when they were in the new project/subprojects).

Observations identified more than one day ago (when they were in the previous project) will not be pushed again to these project/subprojects.

New project and subprojects for observations of Arthropoda without identification.

Links to all projects at the top of this journal post.

1 Like

I think it just worth pointing out that jeanphilippeb routine for flagging unidentified observations is superb and is about 98% correct for Proteaceae (I cannot assess false negatives). It has rescued hundreds of observations from obscurity (https://www.inaturalist.org/observations?project_id=152984&place_id=any&verifiable=any&captive=any).

It would be great if the routine would review all observations above Family in Plants.

3 Likes

Apologies for a batch of - Not an Unknown protea - notifications. About 50, didn’t count.

I tidied up the very few, where I could.

I agree that jeanphillipeb’s projects are very helpful, although I’m not seeing quite as high a rate of correct IDs. But many of the incorrect ones are observations where the observer submitted several organisms as one observation, or there’s an insect or spider lurking amongst the plants, or various other similar situations. Also, it made it easy to pull out the observations of humans and whisk them off into Casual-dom!

1 Like

Thank you all for your feedbacks!

In 2 days I will have finished to treat all “unknowns” that “needs id” and that are older than one month (to let time for an ID from the observer).

If you are curious about how many observations have been treated so far, check these umbrella projects : Taxa A-I, Taxa J-S, Taxa T-Z.

This is not out of reach. It represents 2,500,000 observations to treat:
https://www.inaturalist.org/observations?quality_grade=needs_id&lrank=epifamily&taxon_id=47126

I can start with Southern Africa, representing 50,000 observations to treat:
https://www.inaturalist.org/observations?quality_grade=needs_id&lrank=epifamily&taxon_id=47126&place_id=7105,8489,7140,6986

As many high rank observations have already been treated in Southern Africa (in search for candidate observations for the trees projects 1 and 2), only 9,500 observations are still to treat (to be completed in 2 days) for S.A. :
https://www.inaturalist.org/observations?quality_grade=needs_id&lrank=epifamily&taxon_id=47126&place_id=7105,8489,7140,6986&not_in_project=156653,156655,156657,154158,154159,153322,154354,153984,154232,154233,154271,155743


My next technical objective is to be able to propose the software for being run by several persons, in order to push collaboratively the observations to the projects. I have been working actively on this for several weeks. The question is: who is going to run the software several hours per day, to copy/paste a new API token every morning?

At the present time, the software has no more local observations cache. This was an important technical step. Everything is now managed via iNat server, using the phylogenic projects and several exclusion list projects, so that all observations already treated in one way or another can be filtered out by iNat server itself.

I have to ensure that the sofware becomes robust enough before asking for a collaborative work, an important perspective for the 2,500,000 plants observations to treat, as well as for a bioblitz if you expect a quick treatment. The latest issue I had to manage was related to the fact that an API search for observations also returns observations from accounts that have been suspended.


The only manual action I do is adding users to the “opt-out” and the “does not allow other people to add their observations to projects” collection projects. (The software writes in a log file the users to add to these projects). Even in case of a future collaborative work, it is enough if only one person takes care of that.

4 Likes

Most of these 9,500 observations are being pushed to high rank phylogenetic projects.

An example of a high rank IDed observation pushed to a low(er) rank phylogenetic project:
https://www.inaturalist.org/observations/144139027

Presently identified as Order Lamiales (Community Taxon), it has been pushed to the Family Pedaliaceae project. This looks good, as this observation has 2 IDs Genus Sesamum, and this genus is actually in the Family Pedaliaceae.


BTW, @dianastuder , this observation is also an example of a pre-maverick. I hypothesize that this suggestion from @tonyrebelo will help finding pre-maverick observations.

Toward a Pre-Maverick project? With an URL for identifying the pre-maverick observations in the project, filtering out observations that are have become “Research Grade”.

It would be simpler (again) if iNat could provide a search for pre-maverik observations (observations that can, with one more ID, become Maverick). New feature request (impacting the database)? Who will use it?

3 Likes

People who are checking and resolving their mavericks already.
I certainly found my first 142 pre-mavericks interesting to work thru.

Glad to be able to pre-empt unnecessary work for identifiers. Where iNat is presenting my (old) ID as Diana disagrees with all the (newer) IDs, when I haven’t even SEEN them yet.

1 Like

I like this project a lot and I am curious how you got the API to find those observations.

1 Like

He built that use of the API himself :heart_eyes:

Algorithm for finding the Pre-Maverick observations

6 Likes

Wow! Amazing

2 Likes

I just went through Wyoming, Montana and Idaho pre-mavericks. There weren’t many, but I was able to help around 30% of them. I don’t think I would have found them any other way. Thanks!

3 Likes