API Documentation: Getting private Observation location data for own observations on localhost

General
1

I’m working locally on an app to download my observations from iNat using either the original or the “new” v1 API — I don’t care which. I’m using my valid iNat app_id and app_secret, and authenticating with my own username and password, to get an oAuth token.

In the old API:

If i try the GET request for my observations using a valid token in Postman, I can get the actual location coordinates for my own obs, even when the location is marked private. But if I try the request from Ruby restclient on the command line from my localhost, the location data is omitted from the response.

In the new API:

The location data is omitted in the response in either case.

It’s not documented whether the new API even returns private location data, although I can tell from the response in the first case that the older API does in fact give both public and private lat/lng in separate fields.

Am I missing something? What’s the right way to do make this request, and particularly to make the request from localhost, where it’s not a URL with a valid SSL certificate?

2

i don’t see an app associated with your user_id. what exactly are you trying to do? why do you need an app to accomplish whatever you’re trying to do?

4

if you plan to use the /v1 API, once you get the oauth access token, ideally you should exchange that for an API token (JWT). in some cases, the access token will still work when making requests via /v1, but you’re supposd to use the JWT.

without knowing exactly what your request command looks like, i can only guess that you’re making a request that passes authorization information in an improper way.

i would suggest reading https://forum.inaturalist.org/t/error-in-api-recommended-practices-jwt-instructions/20229 and referenced links, and then come back if you still have questions.

GET /v1/observations will return private_location and private_place_guess when you make an authorized request. the field that contains the “private” version of accuracy is positional_accuracy, and that is returned regardless of whether or not you make an authorized request. (i don’t know why accuracy is handled differently.)

1 Like
5

Thanks @pisum for the response. I can separate my issues here:

I. Using Postman via their website. All of the following seem to work fine:

  1. Getting an access token:
POST https://www.inaturalist.org/oauth/token?client_id={{app_id}}&client_secret={{app_secret}}&grant_type=password&username={{username}}&password={{password}}
  1. Retrieving observations via the old API, with the access_token returned above. This DOES return private_latitude and private_longitude :
GET https://www.inaturalist.org/observations.json?user_id=6321993&extra=fields,identifications
headers: { accept: :json, bearer: {{access_token}} }
  1. Getting observations from the new API v1, using only the access_token above. This returns observation data but does NOT return private_location:
https://api.inaturalist.org/v1/observations?user_id={{my_user_id}}
headers: { accept: :json, bearer: {{access_token}} }
  1. Getting a JWT also works fine:
GET https://www.inaturalist.org/users/api_token
headers: { bearer: {{access_token}} }
  1. Getting observations via the new API with a JWT works ok, but also does NOT return private_location. Here I believe i’m making the authorized request correctly, and passing the JWT correctly, because i’m getting a response with all my observations — the response just doesn’t contain any private_location, nor any location info at all for my observations where the location was marked private.:
https://api.inaturalist.org/v1/observations?user_id=6321993
headers: { authorization: {{jwt_api_token}} }

II. Using RestClient cli (ruby) from localhost

This does NOT return private obs data even with the old API, does not return anything private via the new API v1, and does not grant the JWT period, even though i’m doing it I believe the same sequence as above (see below).

I kind of suspect that this may be because it’s coming from localhost, while the Postman requests DO work because they’re coming from the Postman website with SSL? Is that possible?

Both @JoeCohen and I have been trying to authenticate from localhost by the book, and we are not able to get the private data on our own obs.

irb> access_token_request = RestClient.execute(method: :post, url: "https://www.inaturalist.org/oauth/token", headers: { params: { client_id: {{app_id}}, client_secret: {{app_secret}}, grant_type: :password, username: {{username}}, password: {{password}} }, accept: :json })
=> <RestClient::Response 200 "{\"access_token...">
> # Response contains access_token

irb> observations_old_api_request = obs = RestClient::Request.execute(method: :get, url: "http://www.inaturalist.org/observations.json", headers: { params: { user_id: 6321993 }, accept: :json, bearer: {{access_token}} }) 
=> <RestClient::Response 200 "[{\"id\":2354...">
> # Response contains observation data but no private_latitude

irb> jwt_request = RestClient::Request.execute(method: :get, url: "https://www.inaturalist.org/users/api_token", headers: {accept: :json, authorization: { "bearer": {{access_token}} } })
irb(main):058> 401 Unauthorized (RestClient::Unauthorized)

TL;DR: The only way i’ve succeeded in getting private location data is via Postman’s web interface using iNat’s old API.

Nothing I have tried retrieves the private location data from my local dev environment, even when the responses indicate all auth is 200 OK and both access_token and JWT have been granted.

6

i’m not a Ruby person, and i’ve never used RestClient, but it looks to me like you’re not passing the Authorization header properly. fundamentally, you want your request headers to include one of the following, depending on which token you’re using:

  • Authorization: api_token
  • Authorization: Bearer access_token

so, again, i’m not a Ruby person (and i don’t know proper Ruby snytax), but i would try changing the syntax of the header part of your requests. for example:

  • RestClient::Request.execute(method: :get, url: "https://www.inaturalist.org/users/api_token", headers: {accept: :json, authorization: "Bearer #{{access_token}}" } })
  • RestClient::Request.execute(method: :get, url: "https://www.inaturalist.org/observations.json", headers: { params: { user_id: 6321993 }, accept: :json, authorization: "Bearer #{{access_token}}" })
  • RestClient::Request.execute(method: :get, url: "https://api.inaturalist.org/v1/observations", headers: { params: { user_id: 6321993 }, accept: :json, authorization: {{api_token}} })

i don’t know what Postman is / how it works. if it’s a website, i’m surprised that it would be able to execute a request that would take advantage of your iNaturalist cookies. if it’s some sort of application or browser extension that’s going through your browser and executing requests, then maybe that’s how it’s getting a JWT and private locations from the deprecated API.

7

Thanks @pisum once again.

Postman is a web sandbox for testing any kind of REST API request, sort of like JSFiddle for APIs. You store the basics like your user name, password, API key, secret, etc, as environment variables. It formats the request and headers correctly so you can experiment with the responses — it does handle getting, storing and renewing tokens daily, so it’s keeping them in either a cookie or my account (probably a cookie).

If the Postman authenticated request to the new API gets a 200 response including every other Observation field, but can’t get private_location, it seems a bit more likely to me that the newer iNat API v1 is just not actually providing private_location in authenticated responses, for observations where the location is marked private (not obscured). Are you absolutely sure it does?

These are the indications that my authentication in Postman is A-OK.

  • Certainly the first oAuth access_token is OK: the request for a JWT using that token does successfully receive a JWT
  • requests to the older API for my own observations do return observation data with the private_latitude and private_longitude.
  • requests for my own observations to the new API using the JWT get a 200 response with every field I can imagine except private_location. Responses from the newer API are not reporting any authentication errors. I’d be happy to paste a whole response here.

On the RestClient side, it’s possible that one of my requests might be incorrectly formatted, but they are correct according to RestClient docs.

  • requests for an oAuth token do succeed 200
  • authenticated requests to the old API for observations.json succeed, but do not return private_latitude
  • correctly formatted requests for a JWT (using the oAuth access_token with a bearer header) get a 401

This suggests to me that the originating URL is not ok. Does localhost need to be permitted in our API app?

8

i don’t know what else to tell you besides that you’re probably not passing the Authorization header correctly in your request headers.

you can make GET requests to /v1/observations and observations.json without passing a valid Authorization header, and they will succeed and return results without private location. if you pass a valid Authorization header, you will get the private location fields.

i can’t tell you what the correct syntax is exactly for making a request with a valid Authorization header using RestClient in Ruby because i don’t use either one of those. but you should try my syntax suggestions from the earlier post if you haven’t already.

from https://github.com/rest-client/rest-client, the example GET request that includes an Authorization header is:
RestClient.get 'http://example.com/resource', {:Authorization => 'Bearer cT0febFoD5lxAlNAXHo6g'}

so to get a JWT, i would suggest:
RestClient.get 'https://www.inaturalist.org/users/api_token', {:Authorization => 'Bearer #{access_token}'}

9

Thank you @pisum — That resolved one issue! The JWT request in ruby needs to have Bearer in the string under authorization, unlike the original token request where the auth is in the first level of the header under bearer (and not authorization).

# this is using the `get` shorthand, with more contemporary ruby syntax:
jwt = RestClient.get("https://www.inaturalist.org/users/api_token", headers: { authorization: "Bearer #{token}", accept: :json } ) 
# or using the core class method `execute`:
jwt = RestClient::Request.execute(method: :get, url: "https://www.inaturalist.org/users/api_token", headers: { authorization: "Bearer #{token}", accept: :json } )

Both the above receive a JWT in response, and using the JWT in a subsequent request gets a 200 response.

observations = RestClient.get("http://api.inaturalist.org/v1/observations", headers: { params: { user_id: my_user_id }, accept: :json, authorization: jwt })

However, the above request for my own observations from the new API, using this JWT, again does not get any private_location field, and location is null. This is the same result as Postman, which I believe is reliable.

However, it’s certainly true that passing an invalid JWT will get me a 200 response, with all the observations. That’s unexpected for me. Still, I have yet to be convinced the new API in fact returns the private_location field to the authenticated user who created the observation, when the location is marked private. It seems very unlikely Postman is mishandling this request.

10

i’ve noticed that the /v1 API sometimes seems to do some strange caching. add ttl=-1 to your reqest parameters and try again (or just wait a few minutes and try again later).

11

Will do. Thank you!

12

(update for @pisum) Confirming that ttl=-1 did not do it. Requesting from url:

https://api.inaturalist.org/v1/observations?user_id=6321993&ttl=-1

In the results, my private-location obs, the second most recent, again has no private_location field. It does show the following:

"oauth_application_id": 3,
"geojson": null,
"obscured": true,
"geoprivacy": "private",
"location": null,
"mappable": false,
13

Using the header formatting you suggested, with the old API, I am now able to get private_latitude in RestClient.

obs = RestClient::Request.execute(method: :get, url: "http://www.inaturalist.org/observations.json", headers: { params: { user_id: 6321993 }, accept: :json, authorization: "Bearer #{token}" })

I’m inclined to cut my losses and just use the old API.

14

did you pass the Authorization header when you made this request?

if Authorization: jwt didn’t work, you could try Authorization: Bearer jwt. i don’t think it should make a difference with the JWT, but who knows what Ruby / RestClient are doing?

if /v1/observations isn’t returning private location, all i can suggest is to try different syntax to pass the Authorization header.

when i get my own observation details using a JWT through a Python / Javascript backend, it gets private location just fine.

1 Like
15

@pisum - Thanks! Yes i did, but…

The thing I was missing turned out to be that I have to parse the JWT and just take the value of api_token, which everybody else probably knew by heart since 2017, but I did not. I thought I had to pass the entire stringified JSON key-value pair to authorization: "Bearer {jwt_stringified_key_value_pair}".

No, it has to be authorization: "Bearer {jwt_api_token_value}".

I am getting the private_location now.

16

@pisum’s original answer turned out to be close to the correct solution, but I didn’t understand at the time that I needed to parse the value of api_token, and @pisum correctly discovered that for RestClient, it also needs to be interpolated in a string starting with "Bearer #{api_token}", as below:

RestClient::Request.execute(
  method: :get, 
  url: "https://api.inaturalist.org/v1/observations", 
  headers: { 
    params: { user_id: {{my_user_id}} }, 
    accept: :json, 
    authorization: "Bearer {{api_token}}" 
  }
)
17

@pisum, one question. What is the JWT doing security-wise, if it’s only an extra token that you get with the oauth token? It seems like by that logic, there could be n+1 tokens that you keep having to use the previous token to get — in other words… a little absurd. I understand that React devs are used to JWTs, and I read the note in the docs that there are plans to make JWT access a single step. But i cannot detect any added security from the JWT part of the flow.

18

staff are probably the best ones to ask about why they chose to use JWT for /v1 instead of OAuth tokens. i would just assume that it’s probably just faster and simpler to use JWT. the tradeoff is that JWTs can’t be revoked by the server, but that is mitigated in the case of iNat’s JWTs by having them expire after 24 hours.