iNat API: searching occurrences at places return different results

[ulrich]

Hi all,

from the iNat widget, I know, that the species Spatula cyanoptera has been observed at this location. This is the place:

https://www.inaturalist.org/places/rio-lluta-desembocadura-ap-cl

This is the widget:

https://www.inaturalist.org/places/guide_widget/rio-lluta-desembocadura-ap-cl

The place_id is 131950:

https://api.inaturalist.org/v1/places/131950

https://api.inaturalist.org/v1/places/autocomplete?q=rio-lluta

Now, I am querying the INat API to search for this species Spatula cyanoptera at a couple of given locations, including this said place and have these findings:

1. When searching for observations only at this specific place_id, the results return a reference to the place_id (in the subobject “place_ids” of the returned json)
   https://api.inaturalist.org/v1/observations?q=spatula&place_id=131950

2. When searching for observations on multiple place ids, the results do not (or might not) return a reference to this specific place id (in the subobject “place_ids” of the returned json)
   https://api.inaturalist.org/v1/observations?q=spatula&place_id=132018,132012,132011,143905,132009,131951,132005,132007,131950,142057,142056,146617,146619,132008,132006,143903,132014,132014,132013,132024,132022,132020,132028,132027,132026,132025,141616,141560,141558,141557,141703,141702,141700,141699,141794,141793,141814,141792,137412,141790,141705,141819,141825,141817,141820,141818,141828,141827,141824,141823,141821,141815,141832,141831,141829,141842,141841,141840,141839,141838,141836,141834,141847,141846,141845,141843,141844,141848,141858,141857,141854,141874,141860,141876,141880,141883,141886,141892,142026,141900,141895,132019,142055,132031,131949,131937,132033,132032,142054,132035,142053,142052,132036,142016,142015,142017,141986,142031,142027,142025,142043,142042,142041,142039,142038,142037,142035,142032,141906,141974,141973,141970,141977,141982,141902,141984,142022,142020,142044

(I am not posting the JSON responses here as they are very long but you can easily query those urls from the browser and search for 131950 in the resulting json response)

Is anyone able to explain, what is happening here? Are my queries wrong or is my latter query something that is not understood by the API or is this an API internal issue?

(Earlier I had posted about whether the GET query might fail with a lot of place ids. Maybe this is kind of this issue that the iNat server only receives part of my place ids?
https://forum.inaturalist.org/t/inat-api-search-species-occurrences-on-a-set-of-places/31866/2)

Many thanks for any suggestions.

[pisum]

@ulrich

When searching for observations on multiple place ids, the results do not (or might not) return a reference to this specific place id (in the subobject “place_ids” of the returned json)

i think this is simply because responses from /v1/observations are limited to 30 records by default. to get more than the first 30 records, you could get additional pages using a &page=[page number] parameter and/or increase the number of records returned per page by explicitly setting the &per_page=[up to max 200].

so for example, you’ll note that this will return all 158 records in the set rather than only 30:
https://api.inaturalist.org/v1/observations?q=spatula&place_id=132018,132012,132011,143905,132009,131951,132005,132007,131950,142057,142056,146617,146619,132008,132006,143903,132014,132014,132013,132024,132022,132020,132028,132027,132026,132025,141616,141560,141558,141557,141703,141702,141700,141699,141794,141793,141814,141792,137412,141790,141705,141819,141825,141817,141820,141818,141828,141827,141824,141823,141821,141815,141832,141831,141829,141842,141841,141840,141839,141838,141836,141834,141847,141846,141845,141843,141844,141848,141858,141857,141854,141874,141860,141876,141880,141883,141886,141892,142026,141900,141895,132019,142055,132031,131949,131937,132033,132032,142054,132035,142053,142052,132036,142016,142015,142017,141986,142031,142027,142025,142043,142042,142041,142039,142038,142037,142035,142032,141906,141974,141973,141970,141977,141982,141902,141984,142022,142020,142044&per_page=200

…

@ulrich

This is the widget:

https://www.inaturalist.org/places/guide_widget/rio-lluta-desembocadura-ap-cl

just to clarify a couple of things…

even though the URL includes “guide_widget”, i don’t think this is normally what people think of as the widget, which is an observation-level widget.

this seems to be a component or alternative view of the Species tab on the Place page. if you look at that tab, at the bottom of the pane on the left side, you’ll see a link that will take you to the Place checklist, as shown below:

i believe that the checklist is the source for the the species list shown on the place page. you’ll note that theoretically, you could have species on this list that don’t actually have iNaturalist observations.

[ulrich]

@pisum Many thanks for your explanations, that is really helpful for my understanding.

@pisum

i think this is simply because responses from /v1/observations are limited to 30 records by default.

Great, I haven’t been aware of the pagination, so I need to check how to work around that to include all the results programmatically.

@pisum

even though the URL includes “guide_widget”, i don’t think this is normally what people think of as the widget, which is an observation-level widget.

That’s interesting. I have done this to verify that there might be another widget URL for this place, but it comes up with the same:

• Go to the place page https://www.inaturalist.org/places/rio-lluta-desembocadura-ap-cl
• Click on the Observations tab
• Click on Embed place widget link on the right side of the page
• I get forwarded to the page https://www.inaturalist.org/places/131950/widget where I can configure the widget and end up with the code that contains exactly this link:
  <iframe width="100%" height="600" scrolling="auto" src="https://www.inaturalist.org/places/guide_widget/rio-lluta-desembocadura-ap-cl"></iframe>

Could you say, in which cases there might show up species that don’t have observations for this place?

[pisum]

@ulrich

I have done this to verify that there might be another widget URL for this place, but it comes up with the same:

ah, ok. i stand corrected. i guess the site does treat this and distributes it as its own standalone widget.

usually when folks talk about “the widget”, they seem to be talking about any of the various flavors of the observation widget. here’s and example for your place: https://www.inaturalist.org/observations/widget?place_id=131950.

@ulrich

Could you say, in which cases there might show up species that don’t have observations for this place?

iNat allows folks to add records to a place checklist even if there hasn’t been an actual iNat observation in the place. this is useful for folks who want to keep an accurate list of organisms in that place based on other data sources. for example, for many plants in the US, USDA already has records of counties where those plants have been recorded. so it’s possible to load that data into iNat to reflect that collective knowledge.

i don’t think anyone has manually added such records to the checklist for your place, but there are examples in other places. for example, Deaf Smith is a relatively low density population county in Texas (where there are relatively few iNat observatons). if you look at its checklist, on page 9, there’s a record for a particular dragonfly, Sympetrum internum:

if you look at the record details, you’ll see that it seems to have been created manually, and if you search for Sympetrum internum records in Deaf Smith, you’ll find none: https://www.inaturalist.org/observations?place_id=1549&taxon_id=113514&verifiable=any.