Can't access photo pages from bplant server: 403 Forbidden

Bug Reports
1

I use a lot of images from iNaturalist, and in order to save time on the data entry, and ensure I never make an error on attribution or respecting the license, I have my server parse a single page from inaturalist, the photo page, such as this page:

https://www.inaturalist.org/photos/598865021

I’m able to view the page myself in a web browser, but my server isn’t able to pull it automatically. It is not listed in the robots.txt as an off-limit file…but…I’m also not accessing it with a crawling robot, I’m only accessing it one-page-at-a-time with a script that I manually activate myself by pasting in the URL.

Platform:

URLs (aka web addresses) of any relevant observations or pages: https://www.inaturalist.org/photos/598865021 (or any photo URL)

The web server responds: 403 Forbidden

Oddly, it is still letting the server access and download images if I type all the data in manually. So it’s not like it’s blocked the whole server’s IP, it’s just that it’s not allowing that specific page to be viewed by that server, although I can still view it myself manually.

I really would like to get this fixed ASAP. Not only does having to do the data entry manually slow me down, it increases the possibility I would make a mistake in attribution and/or licensing which is something I care a lot about avoiding. Thanks!!!

My server’s IP is 70.35.204.79 if that helps!

I also notice that the forum’s software isn’t pulling the previews from these URL’s either, so it is like they are being uniformly blocked by any sort of crawlers even legitimate uses.

1 Like
2

For info: iNaturalist is protected behind CloudFlare.

1 Like
3

Yeah, I wouldn’t know what my server is doing that CloudFlare doesn’t like. I don’t see any evidence that the server has been compromised, and it makes very few outbound http requests at all, it doesn’t share an IP with any other business or users other than myself and the sites I run.

It also worked fine for years with this setup and I only first noticed this problem today. Before this I used the feature recently, on March 19th, 2026, and it worked fine then. I haven’t made any changes such as to the user agent or anything else.

4

i don’t understand why you would need to parse this page rather than going through the API.

yes, the bot checks seem to have caused problems with a few things. ex.: https://forum.inaturalist.org/t/social-media-previews-no-longer-work/77159/2

1 Like
5

I haven’t set up the API yet.

This is something I could do but I have limited time. I’m trying to run my websites while also being a stay-at-home parent. Any hoops that I have to jump through are necessarily going to take away from other work I am doing.

I would prefer if this could be fixed some other way like perhaps my IP whitelisted or if there were some simple change I could make server-side to make it work again.

I eventually would like to start using the API, but you know, there are a lot of things on my “eventually” list and I have to be realistic. These days I’m lucky when I get an hour to sit down and work on my website without getting interrupted. Learning to use a new API and doing programming to work with it is a type of task that I do best with large, uninterrupted blocks of time, and who knows when I’m going to get this next.

6

i don’t think this is the sort of thing that companies or organizations usually announce. it’s meant to deter unintended uses of the site like scraping, which is effectively what you’re doing.

using the API would be the normal sanctioned means to get real-time license information.

short of that, i bet the site would allow you to scrape if you passed a header with a JWT with your request, but that seems like more effort than just using the API.

that said, i’m not sure that using the API is necessarily the best answer either. there is no GET /photos endpoint. so you’d have to get photo stuff from the API via the associated observation via GET /observations. that shouldn’t be too bad for 99% of use cases, but i’m not sure exactly what you’re doing on your end and what sort of workflow you’ve created there.

if you’re referencing photos in iNaturalist, i think the best way to do this is actually to copy the photo file into your own server, and keep a record of the observation, the photo id, and the license infromation at the time you copied the image over. then going forward, your site would never have to reference iNaturalist at all. i think Wikimedia does something similar when they get photos from iNaturalist. as i understand it, a bot copies the image to Wikimedia’s servers and also verifies the license at that time. i assume that verification is done via GET /observations, but if it’s doing what you’re doing instead, then their process might be broken, too.

7

The API is definitely the recommended way, but we will set up a temporary solution for the short term that supports your use case. @sylvainmorin will reach out to you for more details.

1 Like
8

I’m a software developer. Updating a website codebase when 3rd library or service makes changes is a normal part software development. It is abnormal for an individual developer to ask a data provider to change their data access policy, especially since you aren’t using the officially recommended ways of accessing their data.

I looked at bplant. I think you are just getting the photo url, license, attribution. That information is available in the observations api.

Here’s a photo page from bplant
https://bplant.org/photo/2422

Here’s the observation api for the observation for that photo. I only included the observation photo data in the request using fields. As long as you have the observation id, you can get the photo data for that observation.

https://api.inaturalist.org/v2/observations/?id=28893727&fields=(observation_photos%3Aall)

1 Like
9

I’m not a normal software developer. I’m someone who maintains a website that is more extensive than ones maintained by whole teams of people, but doing it singlehandedly without being paid a living wage, because I care about the subject material.

I do nearly everything myself: coding, server admin, database admin, GIS, research, writing, manage all social media accounts, responding to emails, corresponding with donors, managing our payment processing system. And nowadays, I am doing all of this while also being a stay-at-home parent of a baby. AND on top of that I also do other work to bring in enough income to comfortably contribute to our household, and I’m basically using that other income to support my work on this site while the donations of it remain modest.

Forgive me if I’m a bit salty about this. I’m doing the best I can, and if I may toot my own horn, I think I’m doing an incredible job. Of course I’m going to fall behind on stuff that basic software developers or system administrators would be on top of in a “normal” environment. Because nothing about this situation is remotely normal. Normally a website like mine would have at bare minimum a team of 3 or so full-time employees working on it.

Oh, and while we’re talking about “normal” let’s talk about the fact that I live in a country that has been descending into authoritarianism and I live in a state and city that has specifically been targeted by the current regime, and besides crap like a friend texting me in the middle of the night that ICE is raiding their apartment complex, we are dealing with massive loss of funding, previously-awarded Federal grants being (illegally) pulled and now caught up in slow lawsuits that are costly, and at the local taxpayer’s (MY) expense, that the state and local governments here are now having a fiscal crises and have dramatically raised property taxes AND utility rates as a result, and of course my wife’s employer is also directly targeted by these evaporating grants as well as all sorts of harassment, things like one of her coworkers who was a legal immigrant, randomly being stuck outside the country for months, now unable to return. And of course this adds additional stress, her job has become more difficult as a result, people in my whole community are now more stressed and more economically strained, so people have a lot less bandwidth and resources to help each other out. And as if that wasn’t enough, our regime just started a new war, after running on an anti-war platform, and the war is further driving up prices, which is hitting our area hard because it’s miserably car-dependent because of decades of broken policy basically forcing car dependency down our throats. And even though I’ve made conscious choices to make my life less car-dependent, the others around me haven’t, I’m surrounded by a culture of short-sighted people who drank the car kool-aid for generations so anyone I hire for anything is now affected by all of this and half my friends don’t even live anywhere near me because there is a housing crisis and they can’t even afford to any more so all of it is just one big fragile mess all breaking because of factors completely beyond my control. Nothing is remotely normal about this situation that I am right smack in the middle of.

It’s frankly a miracle that my site is up and running and I’m still actively developing it, fixing issues as they come up, responding (if slowly) to emails, adding and refining range maps, and doing all the crazy stuff that I do. It’s a testimony to my strength, resilience, and resolve.

I don’t appreciate when people expect me to do more than I’m already doing. People need to recognize that what I’m doing is already going WAAAAY above and beyond what 99.99% of people in our society have a lot more support and a lot fewer responsibilities and still somehow manage to have a lot less productive output. I am very much at the limits of what I am able to do and I am not able or willing to push myself any harder. I don’t know what kind of sheltered, privileged world you live in that you think you can make a remark like that, but it’s clearly worlds apart from the world I live in.

I will get to using the iNaturalist API when I get to it. It’s something I’ve wanted to get around to for a long time. But like I said, it’s something that requires extended uninterrupted time to focus and that is not something I get a lot of today, and I have much more pressing responsibilities when that stuff comes up, like the letter I got saying I made an error on my taxes and owe more money need to get that paid in a few weeks time. That happens first.

10

Thank you! I really appreciate it, it is working now. I would like to get using the API soon.

11

@wy_bio So now I’m trying to set up the iNaturalist API and it looks like the system does not allow you to “create an app” unless you first apply for privileges to create an app, but I don’t currently have this privilege. And when I try to do it, it says:

“In order to submit an application your account must be at least 2 months old and must have made at least 10 improving identifications for other users in the last month.”

In spite of the fact that I’ve made a whopping 21,273 identifications, most of which have been for other people, it only counts ones made in the last month…which doesn’t seem to make much sense to me. But that’s currently how it is set up.

So like…in my case, if I don’t get an exception made for me, I have to find time to sit down and make a bunch of new ID’s and then hope that at some point within the month enough people will respond to those records that some of them will get marked as “Improving” (because this is a condition that depends on the engagement of other users) and who knows how many ID’s I would have to make, I could make 10 and reach it tomorrow or if I happen to get unlucky and no one replies I could make 100 and never reach it.

Do you see how this is difficult?

I am trying to do this “the right way” but it’s difficult. You have to jump through hoops. Maybe they can make another exception for me.

The point I’m trying to make here is that sometimes there are reasons why people do things and it’s a worthwhile thing to go through life being mindful of the fact that everyone is just doing the best they can.

I hate the way I am constantly having new demands and new tasks piled on me. Do this, do that. You have to do this before you do that. OH WAIT this thing broke no you can’t do this thing you have to do these other things first. I hate when people judge me for doing things a nonstandard way. I don’t attack people for doing things weird ways.

Apologies if I’m being aggressive about this. I was obviously very upset by your post.

12

if you’re not making authorized requests, you don’t really need to create an app. all the license information is public. just make your requests without authorization.

1 Like
13

That makes sense they wouldn’t publicize that if it were a deterrent to malicious activity. It’s frustrating because I’ve been noticing an increasing trend of legitimate users of sites being penalized more and more for the behaviors of others, like how everyone is constantly having to solve CAPTCHA’s and jump through other hoops to prove that you’re human. I wish the bad players could just be shut down, ideally with authorities coming to their physical location and restricting their internet connection if necessary, instead of penalizing everyday users.

I already am storing everything only on my server and never making any additional requests to the iNaturalist server.

I care a lot about not placing unnecessary load on servers, both on my own and on others’, and my code is designed accordingly.

The issue is that I like being able to pull the actual photo URL and license info from the URL without having to go in and do a bunch of right-clicking and saving and then re-uploading to the server. I want it to pull the photo directly because it saves both time and bandwidth, since I don’t need or want that photo stored on my local machine.

If there is no endpoint on the API for photos, that is annoying but I think I’m able to work with the API request for the observation listed here.There is unfortunately an extra step because now I need to provide my system with both the observation ID and the photo ID, whereas I would prefer to just be able to paste a single ID in.

It is genuinely surprising to me that the API has not implemented and endpoint for photos. I’d imagine that would be something people who work with the API would have a strong desire to use, as sometimes you are interested in relating something to a specific photo and not the observation as a whole.

For example, the use of 90% of this stuff on bplant is that I have these ID guides and I’m zooming in on a specific characteristic of a plant, like a closeup of a leaf scar or seeds or the underside of flowerheads, or whatever it is that I’m comparing side-by-side. So my site is referencing the photo, not the observation. Also, whenever someone includes a photo from iNaturalist on an external site, it is natural to link to the photo page, rather than the observation page, because it’s a 1-to-1 correspondence with the actual object being shared. I often get annoyed, as a web user, when I follow a link and it leads me to a “collection” page. This is especially the case because some of the best observations that I take the best photos from are ones where the iNat user may have uploaded as many as 10 or more photos!!! So then if you were looking for the original of the one, perhaps to see it higher resolution (I don’t always keep the full resolution if I don’t need it for my uses) you have to click through a bunch of other stuff to find it.

I always want the web to have the most natural structure! I feel like such an old-fogey saying this but nowadays we tolerate too much sloppiness on the web, I want it to be clean and elegant.