Backup: what if iNat suddenly disappeared

I found myself realizing that if iNat disappeared overnight, I would entirely lose my life list of all invertebrates (and get a lot of confusion in vertebrates as my separate lists of them are not up-to-date).

An obvious solution is the CSV download. However this still has the downside of losing everything that has not yet been IDed. Sure, with iNat gone, I could no longer hope to slowly get IDs for everything on iNat, but as I am learning about new groups of animals, I am sometimes going back to my observations myself - or there could be a new similar platform. The problem here is that while I still have all the photos that I uploaded to iNat also stored on my computer, it would be a massive effort to collect them and establish their locations again (some have geolocation metadata, but many donā€™t)

Luckily, the CSV download also provides URLs for the photos - unfortunately it seems that only for one photo per observation and I havenā€™t found any way to change that, but thatā€™s still vastly better than nothing. With a simple bash script, I am now slowly downloading all the images - renaming ā€œmediumā€ to ā€œoriginalā€ in the URL gives me the larger versions and when I just rename them with the observation ID, they are easy to link with the CSV file.

This is a bit of a waste of bandwidth, as I already have all of them, but I simply havenā€™t found any way how to link the existing images to the observations - the upload process removes the filename. In principle, I could tag each photo with an unique string automatically before upload and because the upload preserves the tags, this could be used for linking, but this could have unwanted side effects when managing the photos locally as the proliferation of different tag could crash some software.

Is there any more straightforward method which I am simply missing? The download works nice (should be done in about 4 days and any future update will be much faster, only downloading new observations), but it still feels a bit contrived.

5 Likes

See also:
https://forum.inaturalist.org/t/is-there-a-recommended-way-to-backup-all-my-inat-data/35015/32

6 Likes

If you go to the photos page e.g. https://www.inaturalist.org/photos/337689777

the filename is maintained along with all of the original metadata (itā€™s hidden on photos which arenā€™t yours). You could modify your script to go to that page instead of the .jpg and then scrape the filename. I donā€™t think thereā€™s an API endpoint for the photo metadata.

1 Like

Interesting! This would be a bit more work, because it requires reading the webpage, but would be feasible to do. The main upside is that I would just find the original files and have them in full resolution this way (the downside that it requires thinking as opposed to brute force :))

1 Like

If iNat suddenly disappeared, I would have a lot more free time.

23 Likes

Just on the photo and record side you could upload them to Flickr and import from there. When the ID is made you could then update the file name on Flickr.

Flickr also offers an option to place your images on a map location, and you can group your photos into albums or groups based on whatever criteria you want.

Itā€™s an awkward method, but but it would preserve your images, IDs, and locations.

The paid version of Flickr offers unlimited storage, so unless they change that eventually youā€™re not going to run out of space.

3 Likes

Time for so many more obs toā€¦ oh, wait

1 Like

In the thread linked by thomaseverest above, people note that GBIF does not store independent copies of the images, so that would not be helpful.

1 Like

for what itā€™s worth, since a couple of years ago, iNat photos that have been licensed (not all rights reserved) are stored over in the AWS Open Data repository. i donā€™t know for sure, but i suspect that even if the iNaturalist organization completely collapsed tomorrow, Amazon would still keep the AWS Open Dataset up and available for everyone for at least some period of time, if not indefinitely.

so i would guess that all the iNat observations that made it to GBIF (since they are licensed) would still have the underlying image files available. if GBIF were to go down, too, then you still have the AWS Open Datasetā€™s monthly metadata files. so you could still get basic data associated with the photos ā€“ observer, date, location, taxon.

if Amazon / AWS were to disappearā€¦ well, then half internet would be non-functional. so there would be bigger problems than missing iNat observations / photos in that case.

ā€¦

if you want to download your photos, thatā€™s probably fine, but i think realistically, the main scenario that this kind of backup would save you from is if you (or someone who got access to your account) deleted all your observations (or deleted your iNat account altogether) and iNat staff could not undo this for some reason. or if you wanted to put your data into a time capsule to be opened many decades from now, then i guess this would be another scenario where it might make sense to download everything.

6 Likes

GBIF stores (if only temporarily) at least some photos.
https://www.gbif.org/occurrence/3399299136#occurrencePage_media

1 Like

In way, although it was unintentional, iNaturalist has become a backup of many of my photos. I have a personal rule to not add observations unless Iā€™ve labeled the photo in some way that I can easily search for it on my hard drive. For example:

Screen Shot 2023-12-05 at 6.42.25 PM

Perhaps Iā€™m an exception, but I try to label every photo I intend to keep with a basic name and date. I donā€™t use a mobile phone to take photos either and my tiny point and shoot camera is GPS enabled so all photos are automatically geotagged. If I make an ID mistake, or the taxonomy of an organism changes, or if an expert IDs an observation to a finer level, then itā€™s easy for me to search and locate the photo to rename it.

My personal rule means I sometimes wait weeks or months to find the time to label the photos so I can upload an observation, but Iā€™ve found that I donā€™t have to worry about a website disappearing. Plus if my hard drive and my backup ever crashed, at least some of it would remain accessible on iNaturalist.

3 Likes

Donate, donate, donate so that doesnā€™t happen. I have most of my photos backed up on external hard drives in folders labeled by location and date. It would be a pain to wade through for them though. OTOH, I am more forward looking and my photos and observation skills get better and better. Unless a species Iā€™ve uploaded goes extinct, Iā€™ll keep looking. And fortunately, as a birder, I have full size backups at the Macaulay Library for most bird photos worth keeping. Itā€™s a shame we donā€™t have more permanent repositories for other taxa.

2 Likes

I was thinking the same thing. Iā€™ve started to upload all of my old historical records to iNaturalist or eBird (or both) and when Iā€™m done, Iā€™ll slowly destroy my old field notes especially the ones that were just scraps of paper or local park checklists. Then I was wondering what would happen if these sites closed down? So for that reason I still keep a paper life list of birds, mammals, herptiles which is easy to add to as you donā€™t often add LIFERS. But I otherwise donā€™t track observations of species other than on iNaturalist. If it were to go then having my observation in a box in my office wouldnā€™t help anyone. My kids will discard them when Iā€™m gone. Not only will no one care if I saw Species X on May 17, 1974 at Newman Sound, Terra Nova National Park, Newfoundland but it also has no value if there isnā€™t a public way like iNaturalist to share it. So if iNaturalist goes then so goes my record keeping of seeing recurring species.

1 Like

Yes, a lot more free time for home maintenance and improvement projects. Letā€™s hope that doesnā€™t happen.

3 Likes

Is it because they take up too much space or what? This is not something I recommend.

2 Likes

Re: field notes and collection books. You may find that a state or university herbarium, insect collection, or library would be willing to take your field notes and collection books, maybe photos, as historic records. They all have very limited resources, so this is most likely in the state where most of the records are. This has been done with the collection books that are associated with birds and mammals I prepared as research specimens that are mostly at the museum that decades later took the books.

1 Like

Iā€™ve been thinking about the same thing. https://forum.inaturalist.org/t/is-there-a-recommended-way-to-backup-all-my-inat-data/35015

So simply put, if we could have a method to re-download all of our images from our iNat accounts with the needed information somehow linked to the image, it will act as a iNat backup and solve our concerns.
(iNat often detects scientific names in names of an image and adds it as the ID automatically, same with dates and GPS data- so just like that)

Iā€™d say the needed information would be like:
惻date
惻location and location description
惻observation description
惻ID (preferably with some higher taxonomic names)
惻preferably something that indicates that a series of photos are from one observation- e.g. 100a jpg, 100b jpg),

Furthermore, if this became possible, iNat users can access to their observations OFFLINE! It would be extremely useful.

1 Like

I can already access my observations offline if I keep them because I label them with ID, location, and date. I do NOT want to keep them all or download them all from iNaturalist because that would take more computer memory than I have or want to get, given that I also store large quantities of other files.

2 Likes

youā€™ll probably never get iNat staff working on something like this (unless iNat is near insolvency) because they donā€™t want folks using the system as a photo backup service.

however, itā€™s possible to code something to do what youā€™re describing. iā€™ve already written something that can give you the cURL commands to download photos with filenames prefixed with obs id, seq, date, taxon id, and taxon name. getting location, location description, and observation description would probably require code to actually download the photo for you and then add photo metadata. although itā€™s not super difficult to do this with some EXIF modules out there, iā€™m not going to do it because i donā€™t think itā€™s really worth the time.

(you can read my take on backup above. i donā€™t think thereā€™s much point to downloading photos for backup purposes, except in a few limited cases.)

3 Likes

Sorry to go off-track a little, but do you use HDD or SSD? Those things can bump your memory for relatively low price, so I donā€™t mind having large files nowadays.