Cannot fetch pages beyond 333

Bug Reports
1

Search does not return pages beyond 333, e.g. https://www.inaturalist.org/observations/identify?page=334&place_id=97391 returns “No matching observations” even though there are over 21000 pages of observations from that place.

The problem is with the API, where calling

https://api.inaturalist.org/v1/observations?viewer_id=123&preferred_place_id=7020&locale=en&ttl=-1&reviewed=false&quality_grade=needs_id&page=334&per_page=30&order_by=observations.id&order=desc&spam=false&place_id=97391

…returns this error

{"error":"Result window is too large, page * per_page must be less than or equal to 10000","status":403}

…even when per_page is within limits. Setting the page parameter above 333 will trigger the error.

2

We use Elasticsearch and keep their recommended limit of 10k records, which is why you’re hitting this limit. You can see the GitHub issue here: https://github.com/inaturalist/iNaturalistAPI/issues/134 We should, however, have an error message when this happens, I’ll make an issue for that.

The way to get around this would be to use more filters, such as filtering by taxon, dates, location, etc. For example, searching for all reptiles in Europe that Need ID returns just over 1k records: https://www.inaturalist.org/observations/identify?iconic_taxa=Reptilia&place_id=97391

5 Likes
3

Filtering by dates would be a strait-forward way of progressing through a list past 333. If you check the dates on page 332, then set your new filter to end at that date, your page 333 suddenly becomes page 1 or 2, and you can keep going.

4 Likes
4

I’ve encountered the same issue, but got around using filters. Thanks for posting @mikkohei13, now we know exactly what’s happening.
@tiwane can i suggest to broaden the error message with some type of recommendation like “please, use filters” or something similar? 'cause first time i bumped into this i thought there is some problem with my computer or browser until i cleared my cache and tried it on another machine, only then i figured out it was some kind of a system limit. Thank you for clearing this out!

3 Likes
6

I was curious what my first identification was on here (of the 36742 that I’ve made). But I get an error message when I attempt to look back beyond page 333.

8

This is because of a limit of elastic search (see Tony’s explanation above).

The “fix” is also the same as described above – use search parameters to get to what you want. You can use the API (info here), and pisum has already made a friendly tool for it here. If you’re just looking for your oldest ID, you can use https://jumear.github.io/stirfry/iNatAPIv1_identifications.html?user_login=joe_fish&order=asc.

You can also vote for the feature request to add better searching for IDs.