AI Poisoning on iNat photos - discussion of effects on inbuilt computer vision and implications on reaserch

General
,
1

Hello,

I have recently been seeing more use of so called AI poisoning in artist circles. Many of us especially those that try to take nice pictures probably don’t want our photos being scraped by AI companies to use in their models so I have been considering using AI poisoning software such as nightshade (https://nightshade.cs.uchicago.edu/index.html) and glaze (https://glaze.cs.uchicago.edu/) to poison my images to make them unsuitable for model training.

This however poses an issue and that is that iNat uses computer vision trained on our photos. As I have understood this is not generative AI but I am still concerned that poisoned photos could effect iNats computer vision. There is another tangent and that is that perhaps our observation data will be used by actual scientists for training models that serve a purpose, poisoned images could also damage these models theortetically?

I would be interested if anyone has tested if this software is problematic for iNat’s computer vision and if anyone has any other thoughts regarding this topic, I have yet to use it but I still think its interesting! Perhaps in the future there might have to be a option to note if in image is poisoned so its not used as training data for iNat as to not cause problems. Also the implication in reaserch I think is fascinating as I think many people are happy for there photos to be used in a scientific setting but not so much to line the pockets of shareholders. Will be interested to see what peoples opinions are on this :)

Gustaf

10 Likes
2

FYI: Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI

4 Likes
3

I read the paper and it seems to be ineffective if somone is quite intent on stealing someones art in this case, however it does seem to atleast make it more difficult than no protection at all and in large data sets may not work as not every method has an 100% chance of working? The paper doesn’t seem to mention if its methods work automated in large datasets. Insightful paper though!

4

if you want to approve every use of your photos, you need to either not post them at all or post them without granting any license (reserving all rights).

iNaturalist already has included only licensed photos in the AWS Open Dataset. so that should already disincentivize folks from spending a lot of effort trying to use unlicensed images from iNat in their training sets.

it looks like you currently have your images licensed CC-BY. just in case you weren’t aware, CC-BY does not prevent commercial use of your images.

also, in reality, i don’t think there’s that clear a line that separates scientific use cases and users from commercial use cases and users. so i don’t know if this is the best way to frame your dichotomy of good vs evil.

9 Likes
5

I am aware of the current license I have on my photos, and I am fine with my photos being used without my knowledge, although CC-BY does require attribution which would exclude most AI companies for use in training data technically unless they publish large lists of people who’s data they use which I don’t think that they do. It is clear that copywrite laws are not somthing that concern AI scrapers at the moment so its licensing does not really solve the issue of having photos etc… being used.

Technologically there is no difference in what is being done by scientists and commercial use when it comes to AI, and destinguishing the two without any context is of course impossible. I would say that there is an obvious line in terms of usecase between scientific and commercial uses and intent. I was more interested in the hypothetical regarding how data if “poisoned” may effect scientific data sets if it is not clearly destinguished as such, and how “poisoned” images would effect iNaturalists own CV which I am very happy to have my photos being used to train. Perhaps I was too heavy on the opinion bit of my orginal post when I am more interested in the implication for the CV and in scientific use in general assuming the technology works!

6

This is not the case. There is no requirement that attribution be given individually at training time. Over a 100k CC BY photos have been imported to wikimedia, and those will be used for training without explicit attribution by many, many organizations. Attribution (which is required at time of disseminating results, I think) can legally be met through consolidated or bulk credit lists that are easy to quickly and automatically generate. Individual names will be buried in a sea of others.

7 Likes
7

I agree that the license a user chooses to apply to their content maps fairly closely to whether it is likely to be used for genAI training. Of course, we know that many genAI companies have chosen to take a “So sue me” approach to permissions, but, if a user chooses to apply “All rights reserved” to their images, the work required to obtain the image is sufficiently greater than a bulk download from the AWS Open Dataset that it seems very unlikely anyone would bother to make the effort. When you have a readily accessible dataset of >100m tagged and CC-licensed images, why waste time to scrape the unlicensed ones?

I don’t know that anyone has tried this, but my guess would be that using an AI-poisoning technique such as Glaze on images that are later included in iNat CV model training would cause significant problems, and, to the extent that the technique were to be widely used, would degrade the accuracy of identification suggestions.

There would also likely be an impact on the accuracy of CV suggestions provided for images that have been processed by Glaze. I would expect that ID suggestions for these images would be worse.

I’ll note that the information provided by the developers of Glaze is very much focused on protecting the rights of digital artists, and makes little mention of using it for photographs.

One other thing to think about: iNat guidelines already prohibit most types of manipulated images, and it’s an open question whether an image processed by an AI-poisoning algorithm would qualify as a manipulated image.

5 Likes
8

Interesting! I know some of my photos are used by wikimedia, they all have attribution to me. However as you mention I guess its not clear once it gets further down the line about licensing. Do GenAI companies produce bulk credit lists, doubtful. But then again I really don’t know much about the topic and would rather keep my photos open to be used by people in the end I think! I appreciate your correction on my assumption though.

9

This is true, however as AI models need larger and larger datasets to improve there may come a point in time where hard to get data is the only stuff avaliable, this is of course completely hypothetical though. As mentioned I am more than happy for my photos to be used in almost any other circumstance I just am not a huge fan of the current uses of GenAI and where it is heading, although the technology itself is ofc had no will of its own.

I think this is really interesting and if it does catch on could pose a problem especially if used on species with not a huge amount of observations. I really don’t know how the CV works and if the technology is the same as GenAI in so far as Glaze for example would do anything. Regarding the guidelines I think the intent seems to be more focused on content that is modified in a way that would effect a humans interpretation of the data, I also think it would be quite difficult to enforce it as the images should generally look basically the same theoretically. I agree that the focus of this tech is for digital artists would be interesting to test this! Thank you for the reply!

1 Like
10

I was intrigued, so I tried out Glaze on a very recognizable 1.9 MB photo of a black-tailed deer.

Using the default settings for “intensity” and rendering time, the 4.6 MB “glazed” image that it produced appeared to be identified by CV (via the upload page) just as easily as the original. However, the list and order of other identification suggestions varied slightly between the original image (papillomavirus, dog, rabbit, sheep) and the glazed one (papillomavirus, rabbit, dog, sheep, goat), so clearly the manipulation does have some effect on CV suggestions.

[Edited to add update below]

I then ran the transformation with high intensity and slowest rendering, which was estimated at 32 minutes but actually took more like 90 minutes on my machine, probably because I insisted on using it for work at the same time. The resulting 5.2 MB jpeg shows “marbling”-type textures detectable to the human eye (mine). When I uploaded it via the iNat upload page the top suggestion remained Black-tailed Deer/Mule Deer. The other suggestions this time were papillomavirus, sheep, dog.

I don’t think we can draw a solid conclusion from testing with a single image, but I was surprised to see how resistant the CV suggestions were to this image manipulation. I think it may have helped that I chose a very recognizable image, and I’d expect ID suggestions to be less accurate for glazed versions of lower-information-content images (but also why bother?).

I also don’t think that results from testing the quality of image suggestions for AI-poisoned images necessarily transfer to the accuracy of models trained on similar data. Imagine a scenario where a group of wildlife photographers shoots copious photos of snow leopards (Panthera uncia), and upload many glazed versions of these to iNat. This would likely result in these images making up a significant portion of the training set for future training of the AI model. I would not be surprised to find this model suggesting “snow leopard” as the top ID for glazed photos of regular leopards (Panthera pardus).

Overall, I think the risk that AI-poisoning transformations may degrade the accuracy of CV appears to greatly outweigh the notional benefit that might derive from preventing my images being used to train a model I don’t support.

Here’s another way to view it… What are the likely uses for a large corpus of images each tagged fairly accurately with the name of an organism shown prominently in the image? The only obvious use is to train a model to identify those organisms. Certainly, that model might be built by a for-profit company whose actions I vehemently disagree with. But that’s kind of the bargain we take on by choosing to use a Creative Commons license. And the worst case here seems to be that some nefarious tech giant might also get pretty good at identifying mule deer. I think I can live with that.

11 Likes
11

I was just about to do this myself but you beat me too it! I will try with fungi pictures which the CV is famously very bad at IDing anyway and see if there are similar results!

12

I tested the (metadata-cleaned) original vs. recompressed (lossy jpeg, lossy jpeg-xl) vs. heavy ‘artshield’ protection (an outdated, open-source, crude anti-AI watermark). The first three CV suggestions stay the same (Firstpapillomavirinae/hemionus/virginianus) across the different types of alteration. Only the order (not taxon) of the 4th+ suggestions changed a bit.

2 Likes
13

I’ve never tried glazing photos myself but I have tested taking photos from the internet that were apparently glazed (e.g. from this article) and tried asking AI to make digital art based on the photos. It seemed to be able to do so just as easily as unglazed photos so I wasn’t very convinced that glazing actually does anything. I don’t see why this would be any more or less effective if I had started with digital artwork rather than digital photos.

3 Likes
14

I know nothing about this topic, but apparently, AI poisoning refers to techniques that subtly alter digital images, in order to sabotage machine learning models that are trained on them. They often do this by injecting “poisoned” data that causes LLMs to misinterpret the content.

Nightshade is one tool that makes invisible pixel changes so that AI sees mismatches like dogs as cats. It disrupts generative AI training, while humans see no difference.
​
Glazing, in this context, is using a tool called “Glaze” to protect artists’ styles by adding imperceptible things to images. These changes trick AI models into perceiving a different artistic style, such as interpreting photorealism as cubism, without affecting human viewing.

3 Likes
17

Generating images of those organisms would be a major use-case. Generated wild-life shots, videos, advertising material, images for textbooks, etc. That’s how generated imagery is already being used. I’ve seen medical textbooks with AI generated images of organs and bones, high-end, university-recommended ones too, which is bizarre.

5 Likes
18

I don’t see any reason AI-poisoning would prevent the CV from identifying something in a picture. The target of AI-poisoning is not to prevent analysis of files generally, but to prevent specifically the use of the poisoned files for training gen-AI. The real question is, would AI-poisoned photos cause problems for training iNat’s CV models? I don’t know the answer to that, but iNat’s CV models are not gen-AI but essentially image recognition, so I would hypothesize it might be less of an issue.

2 Likes
19

Good point. Certainly, Gen AI images have appeared in anatomy textbooks, for example (and have been roundly pilloried for their evident fakery).

The economic motivation doesn’t make a whole lot of sense. We’re primarily talking about CC-licensed iNat image content potentially being used as the source here (because scraping all-rights-reserved content would take a lot more work). Anyone looking for a wildlife image or textbook illustration can already use those CC-licensed images directly at no cost. If you’ve read many online or magazine articles about wildlife recently, you’ll probably be aware that iNat images are used this way extensively at present.

So what would someone gain by using AI-generated images based on that same CC-licensed content? Not much really. For any important use, the existing free and accurate images would seem to offer lower cost and less risk of legal or reputational damage.

Most cases I’ve seen of AI images appearing in science textbooks have involved labelled drawings, which would typically cost significant money and time to produce, and I can understand the lazy-and-cheap motivation involved.

I concede that sleazy AI vendors might try to use the iNat dataset to train genAI image models, but I think that a pretty small number of users are going to want to create genAI images based on that training. Why bother, if you already have a huge image database available for free?

6 Likes
20

Why do we think a renowned science magazine uses a farcical AI image of a ‘panda ant’ when they could have used one of the many CC-licensed images (e.g. from WikimediaCommons)? ease of entering a request in a prompt box to get a pretty pic with no intellectual property attached? lack of time to locate and apply the conditions of use for a licensed pic? surely not an economic incentive since both cost nothing (directly) indeed.

3 Likes