Cannot fetch pages beyond 333

Search does not return pages beyond 333, e.g. https://www.inaturalist.org/observations/identify?page=334&place_id=97391 returns “No matching observations” even though there are over 21000 pages of observations from that place.

The problem is with the API, where calling

https://api.inaturalist.org/v1/observations?viewer_id=123&preferred_place_id=7020&locale=en&ttl=-1&reviewed=false&quality_grade=needs_id&page=334&per_page=30&order_by=observations.id&order=desc&spam=false&place_id=97391

…returns this error

{"error":"Result window is too large, page * per_page must be less than or equal to 10000","status":403}

…even when per_page is within limits. Setting the page parameter above 333 will trigger the error.

We use Elasticsearch and keep their recommended limit of 10k records, which is why you’re hitting this limit. You can see the GitHub issue here: https://github.com/inaturalist/iNaturalistAPI/issues/134 We should, however, have an error message when this happens, I’ll make an issue for that.

The way to get around this would be to use more filters, such as filtering by taxon, dates, location, etc. For example, searching for all reptiles in Europe that Need ID returns just over 1k records: https://www.inaturalist.org/observations/identify?iconic_taxa=Reptilia&place_id=97391

4 Likes

Filtering by dates would be a strait-forward way of progressing through a list past 333. If you check the dates on page 332, then set your new filter to end at that date, your page 333 suddenly becomes page 1 or 2, and you can keep going.

3 Likes

I’ve encountered the same issue, but got around using filters. Thanks for posting @mikkohei13, now we know exactly what’s happening.
@tiwane can i suggest to broaden the error message with some type of recommendation like “please, use filters” or something similar? 'cause first time i bumped into this i thought there is some problem with my computer or browser until i cleared my cache and tried it on another machine, only then i figured out it was some kind of a system limit. Thank you for clearing this out!

2 Likes