In pursuit of mappiness (part 1)

(This is the first in a series – hopefully – that describes different ways to create maps using iNaturalist data.)

iNaturalist’s built-in mapping capabilities

iNaturalist provides fairly robust mapping capabilities built in:

Going beyond the built-in tools

But what if you want to visualize something that can’t be done in any of these built-in tools? For example, what if you want to see observations against a map of ecoregions? None of these tools offer a way to choose your own custom basemap.

Luckily, there are several ways to get iNaturalist data into other tools that will allow you produce a lot of different map visualizations. In this tutorial, we’ll cover one of these methods: getting map tiles from iNaturalist’s API to create our own custom maps.

(A future tutorial will cover how to take observations with lat/long data from, say, a CSV export and map them.)

What are map tiles?

One nice feature of online maps is that they are zoomable and pannable. You can view the entire world in one screen, or you can zoom in to see a particular neighborhood in great detail. From one neighborhood, you can pan, or shift, the map in any direction to look at another neighborhood.

When zoomed in, you see only a part of the world. So a website providing a neighborhood-level map doesn’t need to give you information about any other neighborhoods out of view. At the same time, if you pan the map just a little bit, it doesn’t make sense to give you a whole new map if you’re mostly still viewing the same neighborhood. Instead, the most efficient thing would be to deliver only the additional data needed to create the rest of the map that came into view.

Also, it might be appropriate to show different information at different zoom levels. For example, it might make sense to have street labels when viewing a neighborhood vs. continent and ocean labels when viewing the entire world.

So efficient zooming and panning is achieved is by breaking up the entire world into sets of tiles appropriate for different zoom levels, and then delivering only the (additional) tiles needed to cover the area that is in view (or has come into view).

At the lowest zoom level (0), most of the world (excluding a few degrees at the poles) is covered by a single square tile (outlined in red):

Zooming in one level (level 1), the world is now represented as a 2x2 set of tiles:

Note that in this example, the top-left corner of each tile has a label X, Y, Z, where:

  • X = the nth tile - 1, starting from the antimeridian and going east
  • Y = the nth tile - 1, starting form the north pole and going south
  • Z = the zoom level

So at zoom level 1, most of Asia is covered by the tile X=1, Y=0, Z=1.

At zoom level 2, each tile from the previous level is divided in half along each axis, resulting in a 4x4 set of tiles, and so on…

Just for reference, at zoom level 10, much of the city of Paris, FR, is covered by one tile:

… and at zoom level 17, the California Academy of Sciences building (iNaturalist HQ) is covered by one tile:

.

What kind of map tiles does the iNaturalist API provide?

The iNaturalist API provides 3 kinds of polygon tiles and 4 kinds of observation tiles, as summarized below. (There are also 4 kinds of UTF grids that each pair with an observation tile set, but those are beyond the scope of this tutorial and so will not be covered here.)

Observation Tiles

Style: Grid
API Request Template: https://api.inaturalist.org/v1/grid/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/grid/0/0/0.png?taxon=3,630955
Example Response:
Style: Points
API Request Template: https://api.inaturalist.org/v1/points/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/points/0/0/0.png?year=2019,2018&month=4
Example Response:
Style: Heatmap
API Request Template: https://api.inaturalist.org/v1/heatmap/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/heatmap/0/0/0.png?year=2021
Example Response:
Style: Circles
API Request Template: https://api.inaturalist.org/v1/colored_heatmap/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/colored_heatmap/0/0/0.png?verifiable=false&year=2020
Example Response:

Polygon Tiles

Style: Taxon Ranges
API Request Template: https://api.inaturalist.org/v1/taxon_ranges/{taxon_id}/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/taxon_ranges/43694/4/9/8.png
Example Response:
Style: Taxon Checklist Places
API Request Template: https://api.inaturalist.org/v1/taxon_places/{taxon_id}/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/taxon_places/200073/4/3/6.png
Example Response:
Style: Places
API Request Template: https://api.inaturalist.org/v1/places/{place_id}/{z}/{x}/{y}.png
Example Request URL: https://api.inaturalist.org/v1/places/14,50/5/5/12.png
Example Response:

Each individual map tile can be retrieved via a specific API request URL. The request templates noted above have various items in curly braces (ex. {z}) which are replaced by appropriate values when making an actual API request. Additional parameters can be added to the end of the request template / URL to apply additional filters.

Refer to the API documentation for available parameters for each of these API endpoints above. Usually (but not always), the same parameters that you use in the Explore Page can be used with any of the Observations Tile endpoints. The Observations Tiles and Taxon Range Tiles also have a color parameter that can be used to set the color of the markers / polygons (ex. https://api.inaturalist.org/v1/grid/0/0/0.png?year=2021&color=blue)

So suppose I wanted a heatmap of observations observed in India in 2020 and 2021. Then my request template would be https://api.inaturalist.org/v1/heatmap/{z}/{x}/{y}.png?year=2020,2021&place_id=6681. The actual request URL for the tile covering most of India at zoom level 3 (X=5, Y=3) would be https://api.inaturalist.org/v1/heatmap/3/5/3.png?year=2020,2021&place_id=6681, and it would return an image like this:

Let’s build an example map in ArcGIS Online

NOTE: This was originally written just before ArcGIS introduced the modern version of ArcGIS Online. The instructions below are for what is now the Classic version, but the general concepts can still be applied in the current version, although some of the details may vary slightly between the versions.

Now we’ll build an example map, and we’ll use ArcGIS Online (AGOL) for this. Among the many mapping tools available, AGOL stands out by offering a relatively easy-to-use web-based interface, lots of existing layers that you can incorporate into your own creations, and the ability to share your maps easily with others. You don’t need to sign up for an ArcGIS account to create basic maps (if you don’t want to save / share your creations), but since we’ll do some more-than-basic stuff in our example, you’ll want to go ahead and sign up for an account, if you don’t already have one. (The public account option is free.)

Let’s start by figuring out what data we want from iNaturalist. I like pelicans. So I’m going to go to the Explore page and filter for American White Pelican observations, including both verifiable and non-verifiable observations. That gives me the following URL in my browser: https://www.inaturalist.org/observations?place_id=any&taxon_id=4334&verifiable=any.

Most of the time, I will be able to take the parameters from the Explore Page URL (everything starting from the ?) and use those same parameters for my API request template. However, one notable quirk is that the Explore page will filter for verifiable=true and spam=false if these parameters are left out, while the API will effectively get verifiable=any and spam=any if these parameters are left out. So adjust accordingly.

In this case, I want to use the Grid style observation tiles, and I can get data similar to what the Explore Page returned by using the template https://api.inaturalist.org/v1/grid/{z}/{x}/{y}.png?taxon_id=4334. ArcGIS actually uses {col},{row},{level} notation instead of {x},{y},{z}, respectively. So when I’m creating my Pelican layer in ArcGIS, I’ll have to translate accordingly: https://api.inaturalist.org/v1/grid/{level}/{col}/{row}.png?taxon_id=4334.

Now let’s start creating a map in AGOL. Go to https://www.arcgis.com/home/webmap/viewer.html, and make sure you’ve signed in. In the top-right corner of the page, click on New Map > Create a New Map. (If you didn’t sign in, you would click New Map and then Modify Map.) At this point, you should have a blank map, and pane on the left side of the screen should provide an option to Add. Select Add > Add Layer from Web, like so:

When prompted, select A Tile Layer as your data type, and paste in the request template formatted for AGOL. Make sure you also add a title for your layer and some credits, like so:

Clicking Add Layer will give us a map with Pelican Observations:

I’m not a huge fan of the default topographic basemap. So I’m going to replace it with a dark basemap. Select Add > Search For Layers. I’ll choose to search in ArcGIS Online instead of My Content for the term “dark basemap”. The first item returned is “World Dark Gray Base” by Esri. Click on that entry (not on the plus sign in the bottom-right corner), and then when details of the layer slide out, select Use As Basemap:

At this point, I’ll save my map. Select Save > Save, fill in the appropriate information, and then click Save Map:

Now I’ll create a very simple second map that shows lights at night as viewed from space. This will be a proxy for human population with access to the internet. I’ll just start with my Pelican map, and remove the Pelican layer. In the map’s Contents list, click the ... under the Pelican layer, and select Remove:

Once the layer is gone, Add a new layer by searching ArcGIS Online for “night lights”, and then use “Earth At Night” by Esri as the basemap. Click Save > Save As, and then save this map as Night Lights.

At this point, if when you look for the items that you’ve created (https://www.arcgis.com/home/content.html), you should see two maps:

I want to merge these 2 maps together as a “swipe map”. Go to the Classic Story Map’s Swipe Map Builder (https://www.arcgis.com/apps/StorytellingSwipe/index.html?fromScratch), and select the Pelican map as your map, then click Next. When prompted, keep Vertical Bar as your Swipe Style, and click Next. Then for Swipe Type, select Two Web Maps. The Pelican map should be pre-populated as your left map. Select Night Lights as your right map. In App layout, uncheck everything except Enable Description. Then click Open Map.

You’ll now see your Swipe Map with a section to add a description. Add a quick description, then save:

Now you can share your final product with the world by clicking Share. If you want to see the final product, here’s the link to my Swipe Map: https://arcg.is/Wr1S0.

Want another example in ArcGIS Online?

If you want another example to work on, you can try creating this map for Sequoia sempervirens on your own:

Here’s a brief summary of the steps (helpful hints in the next section):

  1. Replace the default basemap with Esri’s World Imagery map
    • (aerial map with no labels)
  2. Add the Ecoregions Level III and IV layer set (by SEGS_GPO) to the map
    • (a predefined set of layers that shows L3 ecoregions at low zoom levels and L4 ecoregions at higher zoom levels, with a color-coded legend)
  3. Add the USGS Hydrology Basemap as the next layer (but not as the actual basemap)
    • (rivers, lakes, and other bodies of water)
  4. Add a Stamen Toner Hybrid XYZ tile layer (see http://maps.stamen.com)
    • (political boundary lines, transportation, and labels)
  5. Add Taxon Range from iNaturalist
  6. Add Research Grade Observations from iNaturalist as Grid style (which is red by default), and then Set Visibility Range for the layer from World to Counties
  7. Add Research Grade Observations from iNaturalist as Point style, setting the point color explicitly to red, and then Set Visibility Range for the layer from Counties to Room
  8. Add a Graticule layer (any will do, but I used the one by ncei_noaa)
    • (latitude and longitude lines)

Here are some hints in case you get stuck:

for #4 above (click to expand)

Stamen Toner Hybrid
https://stamen-tiles.a.ssl.fastly.net/toner-hybrid/{level}/{col}/{row}.png

for #5 above (click to expand)

Taxon Range for Sequoia sempervirens
https://api.inaturalist.org/v1/taxon_ranges/47372/{level}/{col}/{row}.png

for #6 above (click to expand)

Observations for Sequoia sempervirens (grid style)
https://api.inaturalist.org/v1/grid/{level}/{col}/{row}.png?quality_grade=research&taxon_id=47372

for #7 above (click to expand)

Observations for Sequoia sempervirens (Point style)
https://api.inaturalist.org/v1/points/{level}/{col}/{row}.png?quality_grade=research&taxon_id=47372&color=red

Let’s make an example map in QGIS

There are other tools that you can use to make maps. QGIS is a popular open-source GIS application that you can download and use for free. If you don’t already have it, go ahead and download and install it, and we’ll make a map in QGIS.

As before, we’ll start at iNaturalist’s Explore Page and filter for data that we’re interested in. This time, I’d like to look for 2 datasets: verifiable wild gorillas and captive gorillas. I end up with the following Explore Page URLs:

Now I’ll translate those to the following API Request Templates (Point style tiles):

  • wild gorillas1,2: https://api.inaturalist.org/v1/points/{z}/{x}/{y}.png?captive=false&taxon_id=43579&verifiable=true&color=red
  • captive gorillas: https://api.inaturalist.org/v1/points/{z}/{x}/{y}.png?captive=true&taxon_id=43579

Notes:

  1. Remember that you need to add &verifiable=true to the API request for Wild Gorillas, since the Explore Page filters for verifiable=true when a verifiable parameter is not specified.
  2. We’ll make points for Wild Gorillas red by adding &color=red. (By default, vertebrate points will be blue. So red will help us differentiate the two different sets of markers.)

Now open up QGIS. Mostly for our example, we’ll be working with the XYZ Tiles in the Browser pane on the left side of the screen. Expand the XYZ node, and see if you already have a connection to OpenStreetMap pre-defined. (If not, right-click on the XYZ node, select New Connection, and when prompted, add “OpenStreetMap” as your connection name and https://tile.openstreetmap.org/{z}/{x}/{y}.png as your URL, set Max Zoom Level as 19, and then click OK.) Double-click the OSM connection, and that should give you a basemap in the main window.

Now look in the bottom-right corner of the QGIS window. You should see something like “ESPG:3857” there. If it’s not ESPG:3857, you can click on whatever you have, and when prompted, filter for “3857”, and select that. Using ESPG:3857 (aka Web Mercator) will just project the map in the same way as most maps you see online, centered at lat 0 and long 0. (You can use another projection, if you like, though it may lead to unexpected results.) At this point, your QGIS screen should look something like this:

Now we’ll add our Gorilla layers. Look in the Browser pane on the left side of the page, right-click on the XYZ node, and select New Connection. When prompted, name this connection “iNat Obs: Wild Gorillas”, set the URL to the request template we created earlier, and set the Max Zoom Level to 19, like so:

Click OK, and create another connection named “iNat Obs: Captive Gorillas” using the template created earlier and Max Zoom 19. When you have both connections save in the XYZ node, double click each of the new connections to add observation layers to the map.

This is what my map looks like after zooming in a little and panning over a bit:

You may notice that unlike ArcGIS Online, QGIS doesn’t, by default, repeat tiles in an east-west direction, allowing for infinite east-west panning. This brings us back to the concept of projections, which we touched on earlier when I asked you to set your projection to ESPG:3857.

Most XYZ tile sets are designed for ESPG:3857 (exception: GBIF offers 3 additional projections). So, most of the time, when using XYZ tiles for mapping, you should leave the projection at ESPG:3857, especially if you’re mostly interested in higher zoom levels (not viewing the whole world). This will keep things running smoothly.

But if you wanted to see the whole world centered around, say, the Asia Pacific, you could click in the bottom-right corner and select a different projection like ESPG:3832:

Or you may have realized that Web Mercator can really distort distances at low zoom levels. So you could choose a Mollweide projection like ESRI:54009 to compensate:

But note that using other projections can cause things on the map to look distorted. For example, note that in the image above, the points and labels in Africa generally look fine, but the points and labels in North America look skewed.

You can avoid distortions in the points regardless of projection by mapping actual points instead of using tiles, but we’ll cover that in the future tutorial…

Notes about creating maps using iNaturalist’s raster map tiles

(to do)

Additional thoughts and future stuff

This is just the tip of the iceberg in terms of the maps that you can produce this way. If you have any questions, or comments, feel free to discuss below. I’ll try to turn this original post into a Wiki in case anyone wants to make corrections or add/change examples. (I have no special love for the Pelican map. So if you have a more meaningful example map, feel free to suggest that example or add it into the Wiki.) Eventually, when this tutorial seems to cover most of the things that people are interested in, I’d like to turn this into a video and then link to the video in this Wiki. (If anyone wants to take that video on as a project, feel free to do so, if I haven’t gotten to it yet.)

37 Likes

Very cool, thanks for sharing!

3 Likes

This link is missing its destination. Please fix :)

done.

2 Likes

Thank you so much! I used the Stamen Watercolor tiles for my map of Empetrum nigrum:

5 Likes

i do love the whole Stamen collection of tiles. they are quite beautiful and flexible. in some cases, i think the Watercolor tiles can be a little too vibrant. so if I’m using them in ArcGIS Online, sometimes i’ll make them about 70% opaque and then overlay on top World Imagery to darken it a bit and provide ghosts of features like baseball fields and buildings. when i’m working in Leaflet, i like to apply CSS style filters to the Stamen Toner and Watercolor tiles to get all sorts of interesting color combinations. For example, the first few screenshots in the “What are map tiles?” section in the tutorial are Toner labels on top of Watercolor, toned down just a bit by applying an 85% grayscale filter, and i like the resulting coppery land and steely blue water. The fourth screenshot there is Toner Lite with a series of filters applied to turn the white into an olive.

it’s also worth noting Stamen’s inspiration for the Watercolor tiles: Bicycle Portraits - a photographic book / Part III (final) by Stan Engelbrecht / Nic Grobler — Kickstarter.

6 Likes

Thank you very much for the exciting information, I’m looking forward to the upcoming tutorials.

2 Likes

2 posts were split to a new topic: Is there any way to add extra layers (overlays) to the maps on iNaturalist?