-
I don’t think the permissions are set properly yet for the AWS CLI access. @pleary may need to chime in here. But yes the official launch is on April 15th. I’m actually surprised you are able to do anything via AWS CLI given those permission errors I’m seeing. Out of curiosity have you tried https://github.com/inaturalist/inaturalist-open-data/tree/documentation/Metadata/Import/RDS to set this up remotely, because I believe that is working now but its certainly more involved than just downloading the bundle and setting it up locally. Again, pleary will have more information here.
-
We discussed issues with the size of the metadata a bit, including a smaller sample metadata bundle so people can play with it without downloading the whole metatadata. One problem with splitting it up is that its not clear to me that by the different CC licenses is the way most people would want it split. Other’s might want it split by geography or by taxonomy or by date, which is partially the beauty of letting people download all of the metadata to query the subset they want. The alternative RDS setup definitely takes more work to set up and costs a few cents an hour while the database is active. But it does get round having to download the metadata bundle locally and incur the storage, memory and CPU effects of having the DB run on your local machine.
-
We’ve decided to do exactly that for the reasons you’ve discussed. If its not in the version you’ve been playing with it will be in the launch version.
-
Another thing we decided to do exactly for the reasons you’ve discussed. Its not clear we can touch the registry page before launch, but we’re continuing to work on the documentation and will definitely get more info on licenses in that by launch.
-
Correct, we should update the API docs. That’s in there because we incur bandwidth charges when someone downloads images not in the new Amazon Open Data bucket. So that recommendation is meant to help avoid huge costs associated with people downloading huge volumes of photos. That will still apply to people downloading photos not in the open data bucket (unlicensed photos) but we will no longer get charged bandwidth costs when someone downloads from the open data bucket. It will be nice to no longer have to give this mixed messaging around using iNat data (e.g. please use our data for your research / please don’t download our data because we can’t afford it!) and instead just enthusiastically point researchers to this bucket and the associated metadata.
4 Likes