Duplicate prevention: Notify observers if their image checksums match others on the site.

Duplicates are a major problem for both observers, identifiers and project managers.

Every image on the site has a checksum.
Every image a user tries to upload also has a checksum.

If a user is trying to upload a photo which has an identical checksum to an image which is already on the site, you could simply ask the user to double check whether they’re already uploaded this image.

This could prevent a whole lot of unnecessary duplicates.

Sounds like a great idea, as long as it is just a notification, and can be ignored when needed. For example there are sometimes instances where I have used the same exact image for multiple observations of different organisms (e.g., an insect on a flower).

26 Likes

Hey there, not necessarily advocating that you combine threads because they’re a little different but just wanted to make sure you were aware of this somewhat parallel, definitely relevant, discussion/ feature request thread: https://forum.inaturalist.org/t/create-a-way-to-flag-duplicate-observations-and-remove-rg-status-from-the-extras/201

:slight_smile:

2 Likes

Hi Mira, thanks for letting me know about that thread. I agree, it is different though.

If would be good if we could CATEGORISE the TOPICS within the feature request part of this forum!

3 Likes

If you have feedback about the forum, definitely start a topic at https://forum.inaturalist.org/c/site-feedback

2 Likes

We’ve written a spec for the team to review and see if this is feasible, so I’m going to close this topic.

10 Likes

Has any progress been possible for this feature request?

No, there hasn’t.

1 Like

8 posts were split to a new topic: Duplicate observations by the same user on two different accounts?

A post was merged into an existing topic: Duplicate observations by the same user on two different accounts?

https://www.mediawiki.org/wiki/Topic:Tk68mvtbbqwxnh98 On MediaWiki they do have duplicate checks too

SHA1 checksum https://www.mediawiki.org/wiki/API:Imageinfo

2 Likes

Related discussion here:
https://forum.inaturalist.org/t/what-is-the-appropriate-response-action-for-a-user-uploading-multiple-duplicate-images-organisms/3041

1 Like

I’d add that a link to the existing observation should be included in the warning message.

There are valid reasons for uploading the same image (one for a butterfly and one for the plant it’s on, for example), so a link provided would allow the user to determine if they want to continue with the upload or are making an unintentional duplicate.

10 Likes

[quote=“earthknight, post:15, topic:258”]
There are valid reasons for uploading the same image (one for a butterfly and one for the plant it’s on, for example),
[/quote] I do not think it is valid. In that case the image should be reused.

1 Like

It might be months to years later and you’re uploading for different reasons. It’s not realistic to expect everyone to remember every single past photo they’ve taken, plus which ones might be suitable for duplication.

To make it easier for duplication is exactly why I suggested that if this is implemented (which I support in a slightly different form) that it is vital that a link is provided to the observation using the same image. That way the user can duplicate the observation instead of uploading a new image.

8 Likes

I’m not sure how it works and if this is specific to Flickr links, but if I choose a photo to import from Flickr that I’ve already imported earlier, it tells me: “Heads up: this photo is already associated with an existing observation.” I thought that was a smart alert and it would be nice if this could be expanded from Flickr imports to manual photo uploads as well. I don’t know anything about the technical side of how to possibly make it work though.

8 Likes

An efficient way to implement it: compute the checksum (for instance, MD5) of every photo uploaded, store the checksums in a database, compare the checksum of the next uploaded photo with all the checksums already in the database, if there is a match then this photo has been uploaded already.

3 Likes

I like the interface of this app…and it does contain duplicate checking

This app has a nice workflow for uploading photos https://www.wikiuploaderapp.com/features.html
https://download.cnet.com/Wiki-Uploader/3000-20418_4-78600545.html
Although this app is not in “React” i thought the sourcecode was open.(Code is currently under Apache License 2.0; s) .Sourcecode https://github.com/lyrk/Commons and it