Code to extract annotations from exported JSON

if you fetch from /v1/observations, the default per_page value is 30, and the default page value is 1, meaning that if you don’t explicitly specify a per_page parameter in your request URL, you’re going to get only up to 30 observations back from the API, and if you don’t specify a page parameter in your request URL, you’re going to get the first page back.

in this case, the maximum per_page value is 200, and so the de facto maximum page value is 50 when per_page=200 (since iNat won’t return records beyond the first 10000 = 200 x 50 for a given request). to get 10000 records, you have to make 50 requests to the API, incrementing the page parameter value for each request, and preferably waiting at least a second between each request (since iNat doesn’t want you to issue too many requests to the server all at once).

here’s an example in R by @alexis18 that handles this in a for loop: https://forum.inaturalist.org/t/inaturalist-visualization-what-introduced-species-are-in-my-place/12889/10. it doesn’t explicitly issue a delay between response, but i think there’s effectively a delay because the code is executed sequentially.

here’s an example in Python by @jwidness that handles this in a function that calls itself in a way that it effectively iterates until no more results are returned: https://forum.inaturalist.org/t/is-there-a-tool-code-snippet-that-allows-downloading-of-taxonomy-data-from-the-site/14268/7. it also waits an additional 1 second between responses.

in my Javascript-based page, since Javascript can make requests in parallel, i do something more like what alexis18 does because this allows the 1 second delay to be implemented between requests rather than after each response (since there will already be a difference between the time of request and the time of response).

technically it’s possible to go beyond the 10000 limit by incorporating the id_above or id_below parameters, but i don’t think this is a path you want to take except in extraordinary circumstances.

2 Likes