Two questions. 1. It looks like iRecord is meant primarily for UK data. Can and should data from other areas be exported? 2. Does this allow individual identifications (with usernames) on observations to be downloaded? The latter is a feature I’ve wanted for a long time on iNaturalist.
-
No - sorry - I should also make that clear if it wasn’t already.
iRecord is UK only. Please don’t log anything there from outside of UK, it will only clog up their system and they might become somewhat irate with me for posting this up! :) -
Yes, absolutely.
At the moment, there is a “commented out” bit of code within the second cell, specifically for this.
When I update this week, perhaps its useful if I make a more detailed version directed for people from outside of the UK, explaining how it might be used more generally like this. Let me know if there is anything else you want to achieve with it, and I can add that in for the template.
At the moment, the code includes options to get
- all photo URLs at “original” size
- sex annotations
- ids by a specific identifier
- specific observation fields
As well as the more basic stuff you can get from the iNaturalist export tool here on-site…
GPS, location accuracy, community taxon, date…
2 Likes
That sounds amazing! Please let me know if you find a way to allow data to be downloaded from locations outside the UK. Unfortunately, very few species of my group of interest occur in the UK, so this really isn’t that useful to me at the moment.
Its already useable for locations outside the UK - you just choose the location of your choice in the 2nd cell. Have you tried it out ? If not, give it a go ( just skip over the later cells related to converting to grid reference - those are the only parts which are UK specific )
But yes, I will add some more versions shortly to try and make this all clearer … :)
3 Likes
Thanks. that sounds reasonable. I guess what I was getting at was if I can see the exact location of an obscured observation in iNaturalist. either because it’s my own observation, or because the observer has allowed me me to view the obscured location either by joining one of my projects or marking me as a trusted user. Does that trust gets transferred into the download, or if it becomes re blurred. I think from your answer above it does get re blured.
The reason I ask is anything on the IUCN red list is blurred for public view in both systems. but my volunteers allow me to view their obscured locations in iNaturalist. When this goes to iRecord if the locations get re hidden then admins / local record centers using iRecord cant see the exact locations of red list species.
Personal I don’t think its a massive problem. you can always contact the original observer through iNaturalist for critical locations. but if there was a way around it, it would feel more complete.
1 Like
Great to know! I haven’t yet, but will hopefully get a chance to try it out soon.
1 Like
Hello @sbushes. one more thought (that @SteveMcBill highlighted for me (thanks steve) we are collecting observations from many different users (hundreds) into a project. As far as I understand it we have to down load from each user at the moment. If there was a way to download from an iNat project (both collection projects and traditional projects) it would be amazing. It would mean we could download from our national park projects, and potentially from vice-county projects as well.
No idea if that is a possibility or not.
thanks
Pat
Sure. One way to do this is to add a hashtag in front of the user_id line (or delete the line entirely). This will grab all user observations of chosen taxa in chosen location regardless of user ID.
The cell after doesn’t pull in the user_ID though for the spreadsheet itself, so the observations would be nameless - to fix this just add the last line of code here in the same place in the same cell
I will create an updated version soon though (as promised above :) ) where I walk people through using this template as a starting point for more bespoke stuff like this - and go over how to figure out these sort of things. If there’s any problems I can also just save a bespoke version to the Github for you to suit your use-case.
But isn’t it a problem to load data into iRecord on behalf of others? If I am recording in a national park and upload it to iRecord myself …then you grab the same data and upload to iRecord… they will have duplicates of the record… …tricky one maybe.
Thanks Sam. yes. loading on behalf of others is tricky. we will probably ask people to sign up to the project and only include data from people who have signed up. Part of the signing up is to agree to terms and conditions one of which will be user data may be added to iRecord. so it’s opt in.
Would the above work for projects? It looks like it will get all user observations for an area rather than from a particular project?. It may be that some users want some records added to iRecord but not all. in this case they can add just the observations they want to our project which will find it’s way through to iRecord.
We will also have volunteers concentrating on protected and invasive species finding suitable observations to add to our project. (suitable being about image quality, and observers permission) basically a human pre-filter to the data getting into iRecord.
1 Like
Ahh I see. Nice, this sounds great :)
Will have to see if I can make it to some national parks myself to join in!
Looks like you can limit it to a particular project by just adding in a line for “project ID” in the get_all_observations cell like this. :
(Note that the “project ID” is a hyphenated version of the project name)
If you’re interested to know more on how to use this aspect of the code, other parameters you can add to the get_all_observations function to filter the import are visible in the PyiNaturalist documentation here.
2 Likes
Is there a way to make “ident_user_id=” work in selecting observations? I tried it and it wasn’t recognized.
1 Like
What do you want to do exactly nathantaylor ?
If you just want to add a column to the spreadsheet to include IDs from a specific user inside a specific taxon, do the following:
Go to the section “MORE ADVANCED BIT” and take out the initial hash symbols in the following four lines to activate them.
for i in range(len(obs['identifications'])):
if obs['identifications'][i]['user']['name']=="Ian Andrews":
simplified_obs['X Identified by Ian Andrews'] ='I.Andrews'
simplified_obs['Ian Andrews ID'] =obs['identifications'][i]['taxon']['name']
Then the code will just add a column for IDs for a user of your choosing…
(in this case, a user with the name Ian Andrews)
If you would rather use the user login, not the user real name, just swap
[‘user’][‘login’] for [‘user’][‘name’] on the second line
e.g. For the user with the login Ophrys whose real name is Ian Andrews, we would change like this:
for i in range(len(obs['identifications'])):
if obs['identifications'][i]['user']['login']=="ophrys":
simplified_obs['X Identified by Ian Andrews'] ='I.Andrews'
simplified_obs['Ian Andrews ID'] =obs['identifications'][i]['taxon']['name']
(either way though, both code examples above will have the same output in the spreadsheet)
Another example in case that’s not clear!
If I wanted to change to someone entirely different, …say Chris Raper - https://www.inaturalist.org/people/147030 … I could change it as follows
if obs['identifications'][i]['user']['login']=="chrisrap":
simplified_obs['X Identified by Chris Raper'] ='Chris Raper'
simplified_obs['Chris Raper ID'] =obs['identifications'][i]['taxon']['name']
Note also that in these examples, all mentions of the user name other than in the second line are non-specific… they are just whatever term you want for the spreadsheet…so it could just as easily be ‘Identified by expert’… ‘Expert ID’… whatever you like - e.g.
if obs['identifications'][i]['user']['login']=="chrisrap":
simplified_obs['Identified by Expert'] ='The guy who knows about Tachinids'
simplified_obs['Expert ID'] =obs['identifications'][i]['taxon']['name']since this tool uses pyinaturalist to get observation data, this may be something that needs to be handled / updated in pyinaturalist (@jcook).
1 Like
Sure, I’ll take a look. At first glance, I don’t see an ident_user_id param documented in the API (see GET /observations), but maybe it’s on a different endpoint?
1 Like
I can see there’s an ident_user_id defined in the API’s OpenAPI spec, although it’s not referenced in any of the endpoints. I tested it out on the observations endpoint, and it appears to work. I submitted a PR to get that updated in the API docs.
@nathantaylor @sbushes
I just added that to pyinaturalist, so pip install -U pyinaturalist to grab the latest version (v0.12.1).
2 Likes
Amazing: exports all media links, exports annotations - two items I was missing from the iNaturalist export page.
Hopefully not too big offtopic: can community taxon names be exported this way instead of the observation IDs (which I belive is what is everywhere on iNat referred to as “taxon name”)? This is another variable I was missing in the iNat export.
1 Like
Glad it’s of use!
Not off-topic at all. :) It’s a good question.
If you look at the GET observations section in the API documentation, you can see more detail about how you can filter the data.
The stuff in the “Parameters” bit equates to the second block of code in the Notebook.
Here you can see “identified” can be set to true to only pull in “observations that have community identifications”.
I think this will do what you are asking.
If so, you can just add this parameter to the second block of code like so :
Then run through the Notebook again.
Just a follow-up note on this in case anyone is still interested – if you have permission to view private coordinates, you can get them from the API, but you have to make an authenticated call.