Is GBIF not importing data weekly anymore?

From the help page:

iNaturalist generates data for GBIF once a week, and we believe they import it once a week.

The current version of iNaturalist Research-grade Observations seems to be the one from July 25th, 2023. I downloaded the Darwin Core Archive on the GBIF page for the dataset (http://www.inaturalist.org/observations/gbif-observations-dwca.zip), and its metadata indicates it was generated August 1st (2023-08-01 15:00:13 -0700). Does GBIF import the data less often than before?

1 Like

It’s really only been a week since that dataset was posted on 8/1 (currently the 8th), so maybe wait a few more days to see if they pick it up?

My guess is that this is really more a question for GBIF folks than iNat - the iNat data is available if GBIF wants to add it.

3 Likes

For my own records, I’ve noticed it often takes about a month after it reaches Research Grade before the GBIF symbol appears on my iNat record, indicating it’s been added to that database. I don’t know exactly when the upload happens.

1 Like

There is a delay from when GBIF actually posts a record to when that badge appears on the observation on iNat (see some discussion here: https://forum.inaturalist.org/t/how-to-publish-our-observations-to-gbif/43909). So it’s likely that some more recent records without the GBIF badge have been added, but this just hasn’t made its way to iNat yet.

3 Likes

See this, which I just posted:

5 Likes

That’s fair, I can’t find when the previous dataset (generated July 25th) was imported into GBIF but that might even have been a few days ago.

I saw that post but I specifically meant when the data would appear on GBIF. I just finished annotating the life stage of 20000+ shield bug observations and I’m eager to do some statistics, but parsing and filtering 40 GiB worth of observations sounds a bit difficult; I’d rather use the GBIF interface for that.

3 Likes

It seems to have updated August 10th (with the iNaturalist data from August 8th), exactly 14 days after the previous update (July 27th, data from the 25th). So they might import once every two weeks now?

1 Like

Hi, GBIF developer here,

We usually query all datasets (including iNaturalist) for updates 7 days after we last checked, although this isn’t a guarantee — system maintenance, simultaneous updates from several large datasets or very high demand for user downloads can cause small delays. A delay of more than a day would usually be announced on https://www.gbif.org/system-health

The update history for iNaturalist is shown here: https://registry.gbif.org/dataset/50c9509d-22c7-4a22-a47d-8c48425ef4a7/ingestion-history

iNaturalist developers: if you want to, you can send an authenticated, empty HTTP POST request to https://api.gbif.org/v1/dataset/50c9509d-22c7-4a22-a47d-8c48425ef4a7/crawl as soon as the DWCA is ready, and we will process it as soon as possible. The GBIF account you use to make downloads has permission to do this.

5 Likes

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.