Hi all - this is because we’re in the process of making use of the Amazon Open Data Sponsorship Program - here’s some background:
All photos on iNaturalist are hosted on servers we rent from Amazon (specifically using their S3 service). You can think of this as a big bucket of images where each image is identified by a specific public URL. The iNaturalist database keeps track of which photo is filed under which URL allowing the platform to fetch and display photos as needed when you’re using the app or website. As iNaturalist grows, photo storage costs are growing too. We’re currently paying over $10,000 a month in photo storage costs and expect that these costs would double in the next couple of years under iNat’s growth trajectory and current system. This includes the cost of storing the photos (storage) and costs associated with sending the photos over the internet whenever anyone loads a page that displays one or otherwise downloads one (bandwidth).
Luckily, we’ve successfully applied to the Amazon Open Data Sponsorship Program (AODSP) which means Amazon will kindly pick up the bill associated with some of the photos in the bucket, under some conditions. We’re super excited about this because it will help us control costs and, as we’ll describe shortly, will make it a lot easier for groups hoping to access and use iNaturalist data to do so.
The big caveat is that only open-licensed (e.g. Creative Commons) and unlicensed (CC0 / public domain) data are eligible to be hosted in the AODSP. To accommodate this, we set up a new bucket that only contains this open-licensed and unlicensed content, while unlicensed photos remain in the old bucket. We’re also in the process of moving around 70 million licensed photos from the old bucket to the new bucket.
We wanted to let you all know about this now because you’ve already noticed that the URLs associated with photos changing as they are moved to the new bucket. But besides this nothing else should change.
Currently about 66% of iNaturalist photos have compatible licenses and thus are eligible to go into this bucket. Down the road we’d love to encourage people to license their photos if they haven’t already so that these data are more widely usable and so we can take advantage of the cost savings. But since changing your account photo license now triggers a job moving all your photos from one bucket to another please hold off on that for now until we announce this more formally in March.
Another goal of enrolling in this program is to make it easier for others to access iNaturalist photos for research and other uses. Right now this new AODSP bucket is hard to make sense of but we’re planning on including additional metadata to make it a usable standalone dataset. We will start posting a weekly snapshot of associated metadata and provide documentation on how you can make use of these data. We really hope we can build a more visible community of iNaturalist data users around these new features. Keep in mind that nothing is changing about who can access these data, rather we are reorganizing openly licensed photos into their own bucket to make them easier to be used for research, and to take advantage of Amazon’s support in helping iNaturalist offset costs.