Use of AI upscaling

Hot take: :fire: :smiling_imp:

I used to follow insect photographers; there are zillions of them doing mind-blowing macro work.

What I discovered is that each photographer has unique habits and personal preferences when it comes to post-processing.

This was back in the ancient, pre-AI days! :wink:

In my opinion, photography was a lot like painting. Each photographer had a distinct look. Some preferred vibrant photos with boosted saturation, while others tweaked their contrast for a more dramatic effect. They had their own styles in posing insects and choosing angles. They were essentially “painting” with their cameras :art: :camera:

If you were there when the photo was taken, then you would realize that the photo was NEVER an accurate depiction of reality. The photo was ALWAYS a more stunning version of what you actually saw! :exploding_head:

Is AI very different? Could AI be another tool in our creative toolbox? Could AI :robot: even be something to celebrate? :thinking: :smiley:

2 Likes

There’s a huge problem with this, and it’s that iNat is meant to be a website for observing nature and documenting this data for potential scientific use. It’s not meant to be a place for creative or artistic output.

In my opinion, photography was a lot like painting. Each photographer had a distinct look. Some preferred vibrant photos with boosted saturation, while others tweaked their contrast for a more dramatic effect. They had their own styles in posing insects and choosing angles. They were essentially “painting” with their cameras :art: :camera:

The processed photos you saw pre-AI, although edited with color and contrast, would be generally fine on iNat because they’re not adding hairs, legs, or digits, that don’t exist in real life. Thus, users can consult keys and guides for ID and count the number of hairs, digits, etc. to ID to species. The painting analogy doesn’t seem to fit because they’re just tweaking the output from the camera, not painting in new features. It’s true that it could make a more colorful or stunning photo than what it appeared as in real life, but it’s not changing the shape or texture of the subject. It’s emphasizing or de-emphasizing patterns already present in the data.

The difference with this new AI software is that, in many cases (see the feathers people mentioned above in this thread), it adds detail that wasn’t initially there. If someone takes a blurry fish photo and upscales it, it could add fin rays or lateral line scales that weren’t there to begin with. The difference between entire fish species sometimes lay in scale counts or fin-ray counts.
Going off of an upscaled photo, IDers who in good faith believe the photo is legitimate could mark something as one species when it really should be another (or, in the case of blurry photos, left at genus).

Because of this, I think iNat needs to be stringently against people posting AI-generated or AI-edited images. Once those photos are allowed on iNat, the integrity of the site is polluted by fantastical images that don’t depict reality.

17 Likes

I would include color correction in acceptable edits. I have a problem with one of my cameras changing red to magenta. I saw someone else on the forum mention their camera turning blue into purple (or vice versa, I forget). Also removing purple/green fringing. And of course fixing white balance.

6 Likes

I think a DQA of this sort would need to be worded very carefully. The current policy allows drawings as evidence, and I suspect that the criterion of “accurately depicts the organism and surroundings” would be used by those who do not agree that drawings are a valid form of evidence as an argument for preventing such observations from becoming eligible for RG.

An important difference between a drawing and an AI-generated digital image is that the drawing is unlikely to be mistaken for a genuine photograph, and all elements of a drawing normally represent what the observer saw (or thought they saw). So there is at least a basis for evaluating the plausibility of the drawing. With an AI-generated image, we can’t do this. It is essentially a black box unconnected with the real situation in which the organism was seen.

It may make sense to specify that the type of AI being discussed here is what is known as generative AI. I think things like AI-assisted denoising tools would not be considered “generative”.

8 Likes

I fully agree with your points.

I fully agree with your opinion. However, since I mostly take very distant shots of landscapes/with birds/animals etc, I can’t really take more colorful or stunning shots than what I see with my eyes (not talking about close/macro shots captured by a camera/cellphone). So it is evident that no camera lens is capable to capture the true colors and obviously the ambience.
Considering the above, my opinion is that there is nothing to worry about AI color editing (by iPhone/Samsung/any other phone/Canon/Nikon/DSLR/non-DSLR/mirrorless or whatever. They all capture color sheds differently).
Regarding the ADDITION/MODIFICATION issue by AI Upscaling or otherwise: This aspect should not be accepted on iNAT (nor on any social media also).
In my opinion a photo, which may be of a single creature or showing multiple creatures in the frame, does not only depict the appearance of the animal. Photos often/may depict their behavioral patterns, interactions with cubs/chicks/predators, their habitats etc.
If the observation can be made more striking by changing the Background, Interaction, shape, texture or behavior of the subject, the photo will attract many people but we would not like to see such observations (widely accepted by lakhs of nature enthusiasts in other Platforms) at iNAT. Right?

I wouldn’t consider the above point (made by AI) as irrelevant because such photos misrepresent animal behavior to minor users (which is a wrong learning also).


Where is the guarantee that what is being uploaded today in other platforms and accepted by almost all of us will not also be uploaded to iNAT, if there is no barrier?

1 Like

That’s a whole other conversation.

Errata: I should have mentioned here that such color editing does not mean ‘overall color change’ of the creature observed e.g transforming a normal colored crocodile/tiger into an albino, white, black, Golden etc or anything like that for any creature(using AI color edit feature). This is definitely not acceptable.

Topaz AI does not do that. It examines the photograph and enlarges it based on its own content. If one does not attempt excessive enlargement it is quite unlikely to distort a subject. This photo has been cropped from the 8192x5464 raw image to 4764x3086b for æsthetic reasons, and then enlarged in Topaz Gigapixel AI to 7146x4629. Close comparison of the raw photo “screen enlarged” (Command + on a Mac) and the Topaz enlargement reveals no added features, details, artifacts. The two photos are virtually indistinguishable except that the screen-enlarged view is, as expected, not quite as sharp as the Topaz result.

The question “What is reality?” has been debated for thousands of years. Similarly, the question of edited photographs came into being with the first photographs. I have nothing to add to those questions.

I photograph small flora and fauna for æsthetic reasons; I am a very minor contributor to iNaturalist. I guarantee that the subjects that I post will be true (as nearly as jpeg conversions and other technology invisible to me and outside of my control permit) to the raw image that came from my camera. I do not make the same guarantee for backgrounds that contain distracting or incongruent elements.

Addendum: I was not aware when I uploaded my 7146x4629 photo that iNat would downsize it. I can no longer take responsibility for the quality of this photo. The full-size image is at https://www.flickr.com/photos/primeval/53826145904/.

4 Likes

Hmm, what is that ‘own content’? I had a quick look at the Topaz website and couldn’t find any information. Do you have more information about this other than not having seen any problems so far?

Edit: Ah, sorry, I finally get what you mean by ‘own content’. Still, do you have any source for that claim?

2 Likes

This is a particular concern I’ve had as well, and why I’ve shied away from even editing my photos to improve the visual appeal. I’m a big fan of AI but I specialize in a group of flies where the important details are often incredibly minor, and modifying images in such a way that it could affect those characteristics worries me.

If the difference between one species and another is the number of hairs, or even a single spine on the tibia, I could see unintentionally losing or modifying those in some significant degree during the editing process as a relatively easy mistake to make, especially for those less skilled in the identification of a species. This becomes even more worrisome for me if we start talking about using photos to crowd source the location of potentially undescribed species.

6 Likes

People who seem to be drawing a hypothetical line between upscaling and other photo edits … nobody has mentioned photo stacking yet, but that’s common place in macro work and again, one could easily add, subtract or move features unintentionally I think (?)
(though I don’t use photo stacking myself much)

Maybe Krea is a bit of a renegade compared to other upscalers in how it works.
Will have to do more experiments with the others…
But I expect even with the Krea settings on low, one could add definition in a way most people would be fine with.

Drawing a singular line to say one tool good, one tool bad seems pretty arbitrary to me and not possible to implement in a way the community could ever realistically take DQA votes on without a lot of needless debate.

Even if plausible now looking at the Krea image I posted… the AI impacts on photography and editting will only become more accurate, broader in application, more embedded in both camera and editting software…and as such, harder to delineate that which is acceptable from that which is not.

Something to be wary of when looking at observations where something seems off though.

On something like this bird, I very much doubt anyone would give it a second glance before taking to RG had I not explicitly mentioned…given how distinctive and ubiquitous it is at location. Realistically, it would probably get agreed on through thumbnail alone in the identify portal.

4 Likes

This is not an easy mistake to make for an experienced photo editor, regardless of their level of familiarity with the species. I have no difficulty at all avoiding it. There is the practice of using non-destructive editing (Lightroom Classic) on a copy of the raw file so that the untouched raw and the edited file can be zoomed on-screen and compared side-by-side (another LrC feature). For me, the skilled use of LrC and Photoshop tools, in particular careful background selection (i.e., disallowing editing of the subject) and its inverse, subject selection to control brightness and contrast on the subject* are more likely to result in an accurate and easy-to-see presentation of the appearance of the subject than any other technique excepting killing the subject and the use of special lenses and lighting in controlled conditions.

*Even in the field I often make use of professional strobe lighting to reduce harsh, detail-obscuring contrast and to illuminate fine details that might otherwise be lost. That practice frequently obviates the need for any editing beyond cropping.

3 Likes

Yes, my own eye. (I’m 80 y.o. and I’ve only got one good one.) I don’t work slapdash, and I don’t edit original raws, but use virtual copies, which are saved as TIF’s after editing. Then I do a side-by-side comparison of my edit and the untouched raw with screen magnification. If there were anything missing, any artifact added (this can happen if one uses too heavy a hand in editing), any feather or antenna or seta or spine out of place, I would know it. Good enough for scientific imaging.

2 Likes

As I mentioned above I had a very strong and immediate second glance to that image and I think any other detail-oriented bird identifier would have a similar response. However I do agree that someone going quickly through thumbnails could confirm it without looking to closely if it was in a location where the species is common.

But I’m also not sure if it matters that much if it was inappropriately confirmed in a location where it is common? Similar thing if someone draws a generic imprecise bee sketch and it gets ID’d as a common honey or bumble bee species; not a big data quality concern I don’t think. Where it starts to be concerning is if someone is submitting an inaccurate sketch or an inaccurate AI image, and the submitted image implies that a significant new species is being found in an area. I guess this reinforces the importance of collecting specimens for documenting anything important. And right when digital documentation was getting detailed enough that I think a lot of people were wondering why we still do collection of anything…

1 Like

But that only means there are no artifacts in those images you worked on. It could simply be that your images fit nicely into the distribution of the AI training data. From that it is not possibly to draw conclusions on what might happen with out of distribution data.

1 Like

We are not allowed to add links.
But the obs survives as RG, with a taxon specialist ID.
Observer has been suspended and has - according to comments on that post, a history of faking location and date on their obs.

To me, that logic seems broken. At least the comment is a warning (has not gone to GBIF because of observer’s copyright restrictions - so that is good)

1 Like

It sounds like you are using some of these AI-based tools very responsibly, which is commendable! I am sure that many of these tools can be used responsibly and are by some users. I don’t think the tools are inherently bad/evil/etc (like any tools).

BUT I also think that most people using most AI-based tools will be using them fairly quickly and noncritically as a way to “improve” their images. So I doubt that most users will be checking for problems/against originals as thoroughly as you, and that this more common way of using these types of tools could cause issues.

I will note that iNat does have a maximum file size (as you mentioned), so cropping and then upscaling for iNat isn’t a good workflow - it introduces the unnecessary steps of upscaling (and then iNat downscaling) which will lead to noise. Best to just upload the cropped photo assuming it is near to or above iNat’s resolution limit. I suppose for smaller photos, sharpening/upscaling could help. However, in those situations there is “less” image/data for upscalers to work with and they may be less reliable/useful.

3 Likes

Yeah, whatever you want to believe is OK by me.

1 Like

Fully agree - it’s not noticeable in a tiny thumbnail, but the AI messes up some of the details in a way that is immediately noticeable in a full-sized view. The feather placements are off and the photo has an ethereal quality that is so common in AI images. I’m seeing more and more AI birds being used in calendars, advertisements, articles, etc and it is quite frustrating to see, since the AI really doesn’t get all the details right.

Using the AI image vs the real photo might not damage the database in terms of the distribution records being accurate if it is an easy-to-ID species submitted in the correct location, but I don’t see any added value in it and several potential problems mentioned earlier. I’ve seen some cool publications using photos from public databases to reveal new details about geographic variation, new field marks to use for ID, uncover cryptic variation, etc, and while a researcher always has to do some quality control on their image set, that will become harder and harder if AI images are mixed into the database.

7 Likes

What about scorpions, where trichobothria numbers and placement are crucial identifying features? These would likely be altered in an upscaled image. Spider macrosetae are used in identifying at the genus and often species level and these might be changed, even a subtile alteration might be significant.

2 Likes