iNat_patterns — open Shiny app (R) by @andreferrari

Third-party Tools and Apps using iNat API
1

Goal of the tool:
To let anyone see when a species is observed — by hour of day and by month — for any iNaturalist project, place, user or species, without downloading data or writing code.

Niche it fills in the iNaturalist ecosystem:
The Explore page answers where and what, but not when within the day. Time-of-day distribution is visible only by opening observations one at a time or exporting a CSV and charting it yourself. This closes that gap for people who do not work in R or Python. The month chart overlaps somewhat with existing seasonality views, but sits alongside the hourly one so both can be read together under the same filters.

Is there a commercial component, or do you plan to include one in the future? Are donations requested?
No. Free, no account, no donations requested, and none planned. It is a personal project.

What sort of data (if any) does your app collect from its users?
No user data is collected by the app itself. Posit (shinyapps.io) collects server-side usage metrics — session duration and connection counts — for billing purposes, as documented in their privacy policy. No analytics, cookies or logging of queries are introduced by the app code.

Link to your iNaturalist profile:

https://www.inaturalist.org/people/andreferrari

Description:
Pick any combination of place, project, taxonomic group, species, observer and quality grade, and the app fetches those observations and charts them by hour of day and by month. After loading, you can further filter by group, season and moon phase without re-querying. There is also a table and a map.

A few things worth knowing:

  • Queries are capped at 10,000 observations. A live count and time estimate appear before you commit, so you know what you are asking for.
  • Only observations with an exact recorded time appear in the hourly chart. The app reports how many were excluded, since that difference can be substantial.
  • Where an unusual share of records falls between 00:00 and 01:00, a warning appears. Cameras with an unset clock and records whose time was never entered tend to default to midnight, and the API does not distinguish a measured time from a presumed one.
  • Seasons follow the hemisphere, detected from the data rather than assumed.
  • Interface in English and Portuguese, including common names.

Built with R and Shiny, using Leaflet via CDN for the map. Pagination uses id_above rather than page, which avoids the 10,000-record ceiling on the standard paging parameters.

https://andreferrari.shinyapps.io/inat_patterns/

Discussion questions or areas seeking feedback:

  1. The midnight warning is a heuristic — three times the uniform expectation, minimum twenty records. Is that threshold sensible, and are there legitimate cases where a genuine midnight concentration would trigger it wrongly?
  2. Is there interest in overlaying sunrise and sunset times, so the hourly chart could be read against day length rather than clock time? That seems more meaningful for activity patterns but adds a dependency.
  3. The moon phase filter uses an approximate astronomical formula, accurate to about a day. Good enough, or worth the precision?
  4. Anything that would make this more useful to identifiers rather than observers? I built it from an observer’s perspective and suspect I am missing something obvious.

inat_time
inat_time1456×696 25.8 KB

inat_filters
inat_filters1837×865 59 KB

3 Likes
2

i didn’t look at your app or the associated code, but if you’re saying observations are capped at 10000, you’re probably getting data inefficiently.

again, i didn’t look at your app, but just based on your description of it, i would guess that much, if not all, of that functionality already exists in https://www.inaturalist.org/observations/compare.

3

Thank you for looking.

On the 10,000 cap: pagination uses id_above, not page, so there is no API ceiling involved. The 10,000 limit is a deliberate UX choice — above that the wait gets long enough to discourage casual use, which is the audience I had in mind.

On the compare tool: I had not come across it, so thank you for the pointer. Looking at it now, it handles monthly and seasonal distributions well, and there is real overlap with my month chart. What I do not see there is time of day. The hourly chart in the screenshot above — nothing between 07h and 16h, then activity from 17h onward — is not something a monthly view can show, and for camera trap data that distinction is most of what I am after.

If the compare tool does produce hourly distributions somewhere and I have simply missed it, I would rather know than keep maintaining a duplicate.

4

okay. the compare tool doesn’t handle hour of day (although it does handle hour), since the API doesn’t handle hour of day. but if hour of day is important for your use case, i wouldn’t expect it would take much code to add an interval=hour_of_day aggregation option for GET /observations/histogram. i would just make a feature request for that.

even using the existing interval=hour aggregation option, in conjunction with a d1 parameter, would probably be more efficient than trying to get observations and then aggregating observation-level data, unless you’re dealing with date ranges that span more than a decade. look at the response of https://api.inaturalist.org/v2/observations/histogram?d1=2024-07-31T22:00:00Z&interval=hour.

in my mind, using a limit of, say, 10 trailing years, for observation count by hours of day analysis to get data in 1 request is better than trying to get observation-level data 200 observations per request (up to 10000 observations).

5

I tested that endpoint — thank you, I did not know it existed.

You are right about the mechanics: interval=hour returns one bucket per absolute clock-hour across the date range, not a 24-bin aggregation across all dates. Running it without filters over 30 days gives 720-odd hourly buckets, and the diurnal pattern in the numbers is obviously human upload behaviour, not animal activity — it peaks in the afternoon UTC and troughs before dawn, worldwide.

To get what I actually want — one histogram summed across a whole observation history — I would need to fetch that hourly series and then bucket it myself by hour-of-day, same as I do now, just from a smaller number of API calls if the date range fits under d1/d2 without paging. So it would help with efficiency for a long-running query, but the client-side aggregation step doesn’t go away, and I’d still be building the 24-hour histogram from scratch rather than getting it from the API. interval=hour_of_day really is the piece that would make this trivial, if that turns out to be feasible to add.

For now, given the sizes I am usually working with (a single project or place, under a few thousand records), the per-observation approach is simple enough to keep, but this is genuinely useful for anyone hitting the 10k ceiling with a long date range. I’ll file the feature request.

6

you may want to look at https://api.inaturalist.org/v2/docs/ and https://api.inaturalist.org/v1/docs/ to see what’s available (at least officially). the API offers many ways to fetch aggregated data, which will be more efficient than getting observation-level data, especially for large data sets.

and since you seem to be creating a lot of tools, it may be worth looking at https://glauberramos.github.io/inat/ and https://github.com/jumear/stirfry to see some things that have already been made by third parties. these aren’t the only tools out there or even necessarily the best, but i think these are currently the two largest collections of third-party tools out there, and they cover a broad range of different things. so they may give you some ideas for what’s possible.

yes. maybe that’s part of why hour of day was never implemented. also, the system has historically had issues with handling time zones correctly. although many of the problems have been addressed at this point, i think times in the system still can’t be just be completely trusted, especially on older observations. that said, if they’re already providing interval=hour, it certainly doesn’t seem to hurt to also provide interval=hour_of_day.

7

Thank you for those — I had not seen either. Went through stirfry’s file list, and iNatAPIv1_observation_histogram.html is the exact endpoint you pointed me to, exposed as a tool, but it’s the raw series, not bucketed by hour-of-day either. So the gap seems real rather than something I missed by not looking hard enough.

The timezone point is useful and I hadn’t factored it in. That’s a second, more structural problem than the midnight-default one I mentioned earlier — a wrong-but-present timezone would shift observations into neighbouring hours rather than dumping them at 00:00, which my current warning wouldn’t catch at all. I don’t have a good way to detect that from the API response, since there’s no field distinguishing a trustworthy timezone from a guessed one. If you know of one, I’d want it.

Filed the feature request: Add hour-of-day aggregation to the observations histogram endpoint. Included the month_of_year precedent — it already does exactly this pattern for months, so hour_of_day would just be the same logic applied to a different unit — and flagged the timezone caveat above, since it would affect how any such aggregation should be read.