Best way to batch identify

I know there is a feature request (that looks like going nowhere) but I would like to identify about 1k records of a species reverting back to genus. Doing so manually will take several hours of boring, repetitive work. I should probably add a comment to explain myself too, which would take longer. I know this can be done via the API but then I would have to install some sort of development system first, which would take even longer.

Has anyone developed anything to take a list of observation IDā€™s and apply a new ID with a comment?

3 Likes

Maybe you can explain your rationale here, and other people including me would be motivated to help with this repetitive work. I donā€™t think an automated solution exists, or should exist.

5 Likes

Welcome to Identifier Club. ;)

11 Likes

Sometimes taxon changes can help with this.

3 Likes

Depending on exactly what type of automated solution you envisage, it might be prohibited by iNatā€™s machine generated content policy.

If youā€™re just looking for an efficient way to apply a large number of IDs to observations that you have reviewed yourself, then there are various approaches. A well-crafted Identify search, careful use of the hotkeys and an auto-expand shortcut tool can combine pretty well. 1,000 observations isnā€™t nothing, but itā€™s very do-able if youā€™re motivated.

Iā€™m guessing you want to do this because there are ~1,000 IDs for some species that you believe cannot be identified below genus based on photographs. If so, then some version of the above is probably your best approach. If youā€™re trying to address a genuine taxonomic problem then a taxon split or merge might be appropriate.

Share some info here and Iā€™m sure people will have advice.

7 Likes

As for an actual helpful answer- Iā€™d install the iNaturalist metadata tool. It takes a few minutes to work out how to use it, but it can save heaps of time. I donā€™t know if you can set it to do a ā€œdisagreeingā€ broad taxon ID, but you can definitely set it so that one single button on your screen will both add a particular ID and also add a pre-written comment. I use it a lot when going through a genus that has just a couple species options- set one button to add an ID and rationale for each species (an annotations if relevant), then work through the observations and simply click the correct species button for each one to do all the things.
Iā€™d be curious if anyone knows of a way to set it to ā€œdisagreeā€ with previous finer identifications when adding a genus ID though; any time I set it to add a genus ID to something with CID at species, it adds it but doesnā€™t ā€œdisagreeā€ with the species.

6 Likes

Can you link the tool you cited? It might interest me for my long identification courses.

1 Like

https://forum.inaturalist.org/t/announcing-the-universal-metadata-tool-beta/53182

As for my need, I previously identified over 700 organisms to species and led iNat down a spiral of agreements and now realize they canā€™t be IDā€™ed to species from photos so need to be reverted to genus. This will also then stop the CV suggesting the species and me having to make several IDā€™s per day reverting the new ones to genus too. This canā€™t be done through the ID page as most of them are RG.

1 Like

You should be able to edit the filters to include only RG instead of the default which is only Needs ID. Is there some other reason Iā€™m missing that you canā€™t use the identify tool?

3 Likes

Yes, of course, I forgot about this recent change.:+1:

1 Like

As somebody else mentioned, unfortunately while an iNat-based tool doesnā€™t exist, maybe it shouldnā€™t exist. I 100% agree that itā€™s tedious and annoying to explain to hundreds of people why they cannot ID to species from photos, however, if you have a program make blanket IDs, you ignore the possible exceptions, where the person has dissected or done other microscopy to confirm. I personally have IDed many miniscule beetles to species by microscope view, but most photos would not have such sufficient detail, so a tool might be used to automatically suggest genus for this particular species despite having sufficient information to ID to species.

1 Like

This. It might be one of the exceptions where it is IDable. Or the observation may turn out to be of something different altogether and thus a genus ID would not be the correct action. Or there may be other things that need addressing. All of this will get missed by mass IDing.

I think staff have been pretty firmly against tools that would allow for identifying/annotating observations en masse without looking at the individual observations. I agree that it can feel tedious to do the same thing over and over when 95% of the time one particular action is needed. But what about that last 5% (which my experience suggests often includes the most interesting observations)? And one could argue that it is equally tedious to add hundreds of agreeing IDs to common and easy-to-ID species, yet this does not seem to discourage users from doing so.

The whole point of requiring 2 IDs for RG is to (ideally) ensure that the observation has been looked at by at least one other person and reduce the likelihood of errors. This no longer happens if people are using a tool to enter IDs automatically instead.

4 Likes

I had considered the possibility that some might be identifiable (e.g. due to dissection) but if Iā€™m repetitively going through 1000 observations I will phase out and likely not notice such things anyway. Its likely to only be a handful at most.

Then do them in small batches so that you do notice such things. So what if it is ā€œjustā€ a handful? Shouldnā€™t they get reviewed with the same care as everything else? Particularly if the user has gone to the effort to photograph features needed for ID?

3 Likes

While this does not directly solve the original question, hereā€™s a time-saving tip: I use a text expander web browser extension (Text Blaze). These tools allow you to create custom shortcuts that automatically expand into longer, frequently used phrases. For example, the shortcut ā€˜/speā€™ could instantly expand to ā€˜Identification to species is not possible without microscopic examination.ā€™

5 Likes

that handful is just as important as clearing the decks.
More so probably.

1 Like

You keep saying I when you just as well could be saying We. All youā€™d have to do is share the details.

3 Likes

Agreed. This was discussed on the post about the metadata tool, and it was decided that while the tool can be used to auto-generate several actions at once (ie a annotation, comment, and ID simultaneously with one click), the tool can only be used to do this to one observation at a time. So each observation has to be reviewed individually on the identify screen, but if the photo shows an adult female monarch butterfly, a button can be set up that will annotate ā€œadultā€ and ā€œfemaleā€, add ā€œmonarchā€ as an ID, and add a comment explaining how to identify it, all with one click, which is helpful when reviewing lots of the same organism in a row.

But I agree, there should never be a tool that allows IDs to be added en masse to observations that arenā€™t even open. Thereā€™s simply too much relevant information that could be in the observation that isnā€™t visible from the thumbnail- and that information could change the ID one wants to add. Iā€™ve had my fair share of identifiers kick things back to genus that I did dissect to identify, because theyā€™re on ā€œauto-pilotā€ disagreeing with everything. This would be made so much worse if there were an option to mass-identify without even opening the observations up.

if youā€™re just correcting your previous IDs, this strikes me as the least objectionable case for bulk identificaiton. to me, this is not much different than a mass change of your IDs resulting from a taxon swap.

there shouldnā€™t be a need to install anything, if you donā€™t want to. as far as i know, any machine running a modern Windows, Mac, or Linux OS should be able to execute necessary commands in the OS command line via curl.

fundamentally, this would be the necessary request to add a given identification:

curl "https://api.inaturalist.org/v1/identifications" -X "POST" -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: %jwt%" -d "{\"identification\":{\"observation_id\":%observation_id%,\"taxon_id\":%taxon_id%,\"vision\":false,\"disagreement\":false,\"body\":\"%comment%\"}}"

(note that you might set disagreement to true instead of false, depeending on exactly what youā€™re trying to accomplish.)

there are lots of other scripting languages that wouldnā€™t really require a complicated dev setup either. if you want to run Javascript, you should be able to run the necessary code in any desktop web browser console; if you want to run Python, you could do it in your browser via something like JupyterLite or Google Colab; etcā€¦

hereā€™s an example of running curl commands in Windows: https://forum.inaturalist.org/t/bulk-selection-of-observations-to-add-to-a-project/1747/70, and hereā€™s a example of running Python in a notebook via JupyterLite: https://forum.inaturalist.org/t/whats-the-best-way-to-share-python-code-nowadays/48554. hereā€™s an example of Python via pyinaturalist in Colab: https://forum.inaturalist.org/t/how-to-bulk-batch-add-its-sequence-data-to-an-observation-field/31382.

1 Like

The current version of the metadata tool allows users to apply arbitrary sets of actions (eg, a new ID, a comment, plus any annotations or OF values, among other things) to a full page of observations at a time. If you change the per page limit to the max, thatā€™s 200 and you could do 1000 observations in 5 easy actions (although it takes a while for them to actually go through). It is not difficult to install or set up IMO.

I asked Tony directly to tell me whether this was a violation of the machine-generated content policy as Iā€™ve implemented it, and I never heard back.

I am not aware of a way to set ā€œagreeā€ or ā€œdisagreeā€ for contradicting higher level IDs through the API but I will look into it and see if I can make it happen; thatā€™s definitely something Iā€™d like to support.

ETA: I figured out how to add the Disable parameterā€“thanks for prompting me to look into that!