Find frequency of use of observation fields?

General
#1

I neither know a way nor see anything obvious in the API documentation, but maybe others know.

Is it possible to see the list and frequency of use for observation fields for a defined subset of observations ?

For example if I needed to know all the observation fields that have been used, and how often on all butterfly observations in Ontario, is this something that can be done?

2 Likes
#2

an inelegant solution would be to call the set and then to ask for it again but only the ones with the field, and then you’ve got your answer. I know nothing about the API though…

#3

you could build off of this: https://api.inaturalist.org/v1/observations?ofv_datatype=taxon%2Ctext%2Cnumeric%2Cdate%2Cdatetime%2Ctime%2Cdna&per_page=0

if the number of, say, butterfly observations with obs fields is small enough, you could iterate through the observations to determine which observation fields they are using.

bottom line is yes, but i guess it’s just a question of whether the benefit is worth the effort…

#4

It’s not a hypothetical, the use case is trying to help the publishers of Ontario’s butterfly atlas to determine which fields they should include in their extract to populate the atlas and/or assign as recommended fields for the project.

#5

well, I would argue that none of the fields will be used that often, both because most users don’t use them and because there are so many. perhaps some groups of OFs can be combined for your purposes, though that makes your task more complex.
also, a field that is only filled in sometimes, eg. “eating”, will have wildly different use than one that can be filled even in the absence of information or a negative result, eg “has X (yes, no, undetermined)”

#6

does this mean you want (my) help to get the data you’re looking for?

here are some things i can gather initially without doing a lot of novel coding:

  1. there are about 95K Ontario butterfly observations in total: https://api.inaturalist.org/v1/observations?per_page=0&taxon_id=47224&place_id=6883
  2. about 23K of these use some sort of observation field: https://api.inaturalist.org/v1/observations?ofv_datatype=taxon%2Ctext%2Cnumeric%2Cdate%2Cdatetime%2Ctime%2Cdna&per_page=0&taxon_id=47224&place_id=6883
  3. just scanning through a few small sets of these observations**, it looks like the most commonly used observation field is probably insect life stage. it looks like 19K of the 23K observations use this field: https://api.inaturalist.org/v1/observations?per_page=0&taxon_id=47224&place_id=6883&field:insect%20life%20stage. (this sort of makes me scratch my head because people still use this even since life stage annotations were implemented.)
  4. another popular observation field is count (8K of 23K): https://api.inaturalist.org/v1/observations?per_page=0&taxon_id=47224&place_id=6883&field:count
  5. another popular field is nectar plant (2K of 23K):
    https://api.inaturalist.org/v1/observations?per_page=0&taxon_id=47224&place_id=6883&field:nectar%20plant
  6. the rest of the observation fields i see in my cursory scan seem to be much less commonly / consistently applied (i’d be surprised if there were any others that were used in more than a few hundred observations)…

** to replicate my cursory scan of observations that use observation fields, you can do the following:

  1. open https://jumear.github.io/stirfry/iNat_print_friendly_obs.html
  2. use “6883” (Ontario) in the place id input box
  3. use “47224&ofv_datatype=taxon%2Ctext%2Cnumeric%2Cdate%2Cdatetime%2Ctime%2Cdna” (butterflies, plus an extra hack for observation fields) in the taxon id input box.
  4. uncheck all the options except projects and observation fields
  5. click the query button, and scroll through the results (note: the page loads 30 observations at a time, and it will load 30 more records once you scroll to the end of the set.)
  6. to sample other observations, refresh the page, and repeat 1-5, but before step 5, also filter for miscellaneous date ranges

…

so knowing all this,

  1. do you still really want to dig deeper into this?
  2. if so, are you asking for (my) help to dig deeper?
2 Likes