Understanding and exporting annotations

[wamoz]

Some of you may be aware of the Harmful Algal Bloom currently occurring here in South Australia. A Project has been created to collect obs of affected animals - https://inaturalist.ala.org.au/observations?project_id=238172.
Automated additions to the Project require the ‘Dead or Alive’ Annotation attribute on the observation page to be set to ‘dead’.
When exporting observations (from the Explore area), a ‘field’ called ‘dead or alive’ can be selected however this is not the Annotation attribute.
And I couldn’t find any selection in the export page that would show this (or any of the other) Annotations available.
I’d like to a) understand the difference between the two ‘dead/alive’ settings and b) be able to select annotation attributes for inclusion in an export.
Thanks!

[jwidness]

The one that you can currently export is called an Observation Field. These are user created and are sort of a “wild west” situation, with duplicates and misspellings and abandoned fields, etc.

You can think of Annotations as “official” versions of some very specific observation fields. They are standardized, translated, etc. But they’re unfortunately not downloadable from the Export page (see https://forum.inaturalist.org/t/data-users-what-are-your-use-cases-and-requests-for-exporting-data/2972 and some of the topics linked from it for further information).

If all the observations must already have the Dead annotation, I’m a bit confused why you would want that particular annotation in the export file?

[calvertm]

I believe the best way to download observations with a particular annotation would be to go to the Explore page, edit the URL to filter your search results to only observations with a particular annotation, then click on the ‘Filters’ box and click ‘Download’ in the bottom right corner of the filters menu.

To filter for observations with annotation values set to ‘dead’, you would add this to the end of your URL:
&term_id=17&term_value_id=19

So that your full URL would look like something like this:
https://www.inaturalist.org/observations?project_id=238172&term_id=17&term_value_id=19

To filter for observations with annotation values set to ‘alive’, you would add this to the end of your URL:
&term_id=17&term_value_id=18

To my knowledge, the downloaded file will not have a column with the annotation value. You will want to create a column and fill in the annotation value yourself. I think you can only do this for one annotation value at a time. So if you want observations annotated as Alive and observations annotated as Dead, it would be best to download them separately, add a column to each file and fill it with either ‘Dead’ or ‘Alive,’ then combine the two files.

Another way to do this would be to download the observations through GBIF, in which you can filter by annotation values and in which downloaded data has columns for each annotation value. However, not all iNat observations make it to GBIF and it takes some time for eligible iNat observations to sync with GBIF.

Edit: here is a table of all the term IDs for different annotation values, which you would use to edit the URL to filter for a particular annotation. I am not the first person to make something like this, but I didn’t want to go digging around the forum to find where this has already been written out.

Column 1	Column 2	Column 3	Column 4
Annotation	TermID	Value	TermID
Life Stage	1	Adult	2
Life Stage	1	Teneral	3
Life Stage	1	Pupa	4
Life Stage	1	Nymph	5
Life Stage	1	Larva	6
Life Stage	1	Egg	7
Life Stage	1	Juvenile	8
Life Stage	1	Subimago	16
Sex	9	Female	10
Sex	9	Male	11
Sex	9	Cannot Be Determined	20
Flowers and Fruits	12	Flowers	13
Flowers and Fruits	12	Fruits or Seeds	14
Flowers and Fruits	12	Flower Buds	15
Flowers and Fruits	12	No Flowers or Fruits	21
Alive or Dead	17	Alive	18
Alive or Dead	17	Dead	19
Alive or Dead	17	Cannot Be Determined	20
Evidence of Presence	22	Feather	23
Evidence of Presence	22	Organism	24
Evidence of Presence	22	Scat	25
Evidence of Presence	22	Track	26
Evidence of Presence	22	Bone	27
Evidence of Presence	22	Molt	28
Evidence of Presence	22	Gall	29
Evidence of Presence	22	Egg	30
Evidence of Presence	22	Hair	31
Evidence of Presence	22	Leafmine	32
Evidence of Presence	22	Construction	35
Established	33	Not Established	34
Leaves	36	Breaking Leaf Buds	37
Leaves	36	Green Leaves	38
Leaves	36	Colored Leaves	39
Leaves	36	No Live Leaves	40

[pisum]

@jwidness

If all the observations must already have the Dead annotation, I’m a bit confused why you would want that particular annotation in the export file?

yes, you can assume that any observations in such a project are annotated as Dead.

@jwidness

The one that you can currently export is called an Observation Field. These are user created and are sort of a “wild west” situation

i would just add that some observation fields (such as “Dead or Alive”) are linked to specific annotations. if you were to add the observation field “Dead or Alive” with the value “dead” to your observation, that would automatically add an “Alive or Dead = Dead” annotation. similarly, deleting that observation field would remove the annotation. Adding the annotation does not add an observation field though.

it’s also worth noting that annotations the voting mechanism. so even if someone annotates an observation as, say, “Dead”, another person could come along and downvote the annotation. if the net votes for a particular annotation value (including the original annotation) <=0, then that particular observation would be ignored when someone filters observations by that annotation value. (it would also drop out of your project.)

@calvertm

To my knowledge, the downloaded file will not have a column with the annotation value. You will want to create a column and fill in the annotation value yourself. I think you can only do this for one annotation value at a time. So if you want observations annotated as Alive and observations annotated as Dead, it would be best to download them separately, add a column to each file and fill it with either ‘Dead’ or ‘Alive,’ then combine the two files.

if doing this sort of thing, i would download all observations first, and then separately download a set filtered for each annotation value i’m interested in. then use those subsets to populate a column (or multiple columns) in the “allt” set. (this is the easiest way to make sure you’re also capturing unannotated observations, when they exist.)

but for a set that is only ~15000 observations, it’s also possible to just get the observations via the API and parse out the annotations and observation fields you’re interested in. (i have a Juptyer notebook that could be adapted for this: https://jumear.github.io/stirpy/lab?path=iNat_APIv1_get_observations.ipynb, but there are lots of other ways to get this information, too.)

[wamoz]

Thanks for that …
The exercise I did was to search within a boundary for various taxons to find a) obs where the ‘dead’ flag had not been set for a deceased animal, and b) duplicate observations. The second issue has arisen because the community has been ‘activated’ to record dead marine creatures affected by the HAB.