Data users— what are your use cases and requests for exporting data?

if you’re going to be making multiple requests, i would think it’s easier just to do something like this:

the female and male sets in the above example could just be pulled back as ids, and then you could just match them up by id with the “all” set in two extra columns where match would indicate male or female and lack of either match would indicate no male or female annotation.

for folks who know SQL, this might be expressed as:

SELECT A.*
   , [MALE] = CASE WHEN M.ID IS NULL THEN 0 ELSE 1 END
   , [FEMALE] = CASE WHEN F.ID IS NULL THEN 0 ELSE 1 END
FROM ALL A
LEFT JOIN MALE M
   ON A.ID = M.ID
LEFT JOIN FEMALE F
   ON A.ID = F.ID

(this assumes ID is unique within each set.)

2 Likes

To manually add observation fields for the query has already been suggested several times. I am chiming with that, and would like to have the opposite option as well: To delete observation fields from the query.
I once uploaded an observation to a project, where a lot of (very specific) fields were required. I will not use those fields ever again, but they are now taking up a considerable portion in the CSV download form, sitting there forever and cluttering the space, making it harder to find and tick other fields of use.

3 Likes

It would be useful to add taxon-associated data to a species list. I am thinking specifically of any threat-status. A significant use-case is the ability to query the occurrence of any threatened species in a particular area and to download that data for reports/management plans/consent monitoring etc. Perhaps already possible but I’ve not found a way.

2 Likes

Hi Carrie - thanks for this, I appreciate the links and example - but I am still holding out hope for a refreshed download format that includes annotations.
David

2 Likes

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

I’ll second this. There are definitely multiple search terms that work in Explore/Identify that don’t work in the export tool, e.g. multiple place_ids or without_taxon_id.

4 Likes

It would also be great to have the ability to download the URLs for more photos (and at original resolution) added to the export tool. This would be super useful to support UK users to be able to log data on our recording hub, iRecord.

I made an interactive notebook to solve it for now, but long term would be great to have an inbuilt option for this. The notebook might also be of use to some folks on this thread too - see more here.
It has some code to grab annotations …and obs with IDs only by certain users for example, as @nathantaylor suggested …but might also offer a handy starting point for anyone wanting other bespoke data that the API enables but the website does not.

Also, likely a bit much to ask! … but in an ideal world, to have the ability to export as a UK map reference, as I have in the notebook, would solve the other major issue for UK users & iRecord.

1 Like

I’m not certain of this, but I don’t think the site keeps copies of the original resolution photos.

I’m not sure it would fit with their desire to not be used as a photo backup service, and if they have higher resolution photos in cases where users upload them, why incur the storage costs (both financial and technical) of keeping them and not displaying them?

Oh, I just meant “original” as in the best resolution the API offers… there is small, medium, large or original. I think (?)
But yes, maybe its not the original original :) I only upload my pics at web resolution, so I wouldnt notice…

I’d also like to download a species list and came across this discussion posting. Does anyone know if it’s now possible to do this on iNat?

Fields for Downloading Species Spreadsheets: When we EXPORT OBSERVATIONS from iNat there are many choices of fields to include in the spreadsheets. Can iNat add a field for Annotations, specifically plant phenology (flowers buds; flowering; fruiting; no evidence of flowers)? I know most people don’t annotate their observations but some do. Hopefully that will become a more frequent addition for observers and encourage observing and photographing fruiting and other phenological stages. The ability to down load those phenology fields would be very helpful for research purposes.

1 Like

I moved your post about exporting annotations to this data user topic. You can see there has been some discussion about why it’s not so simple to add them to exports (e.g. because there are multiple possible values, because the values can be voted on, etc.), and some good suggestions for alternative workflows, e.g. here.

1 Like

My apologies if this is not the place to post this question (i.e., if it should be a new thread, or if this has already been answered)–I’m really struggling to subset my export requests to keep things under 200,000 and having enhanced flexibility to dictate what to exclude in the export request will be tremendously helpful. Currently in the Explore view, one can exclude one or more place IDs as well as exclude multiple taxa. But when I have the places / taxa I want (and under 200,000 records) and click on Filters → download, the exclusion modifiers disappear and I’m left with an export list that exceeds 200,000 records again.

I understand the desire to not make export sizes unlimitedly large, but providing the option of exporting exactly what you see in the Explore view, if under the record limit, will be tremendously helpful!

Thanks!

While I’m here, another semi-request–is it possible to revisit the issue of not all taxon ranks being exportable? For example, as a bee biologist myself, the most useful higher taxon for ecological analysis is subgenus (which is also often the maximum achievable resolution), but that rank currently cannot be exported as a column. I imagine that researchers working in other groups may have their own favorite rank that would be super useful to them, but not exportable, as well.
Thanks so much for your consideration!
James

1 Like

I have downloaded data from iNaturalist to help with creating a vascular plant checklist for North Bay and area in Ontario. With a bit of programming I can cull the downloaded list to just list observations not in earlier versions of the checklist. This shortened list would be examined carefully to determine if the new species correctly belong on the checklist.

hmmm… if that method doesn’t work, you might be able to set up the exclusions in a collection project, and then export by filtering on that project.

(not applicable in this case, but it might be worth knowing for the future: there’s a similar concept where you can filter by multiple taxa from a list, but i don’t think there’s an exclusion version of this filter.)

Is there a way to download a species list of all the exant species of the world? Not just the observations but a full list of known species.

Yes, this would be great. I raised this on another thread: https://forum.inaturalist.org/t/getting-number-of-photos-in-each-observation-during-data-download-export/37726

Instead of a single output field image_url, I would suggest that the following would be very useful if available in the exported CSV:
image_url_stem – the basal part of the url except for the final slug (e.g., excluding image1.jpeg)
number_of_images: how many photos
image_filenames: a list of file names separated by a pipe operator for instance (as is recommended for GBIF), e.g., image1.jpeg | image2.jpeg

For analysis or downloads of all photos, image_url_stem can be concatenated with disaggregated image_filenames easily.

This will be a very useful feature to add, I feel. Thanks.

That would be a bigger server crash than our friend Gerald.

1 Like