Overzealous Identification

[tallastro]

I think the prize of Research Grade and the leader boards are just too tempting for many. A few ideas to help reduce people gaming the system or even wanting to.

1. Can you use a less exciting term? Consensus? Agreement?
2. Hide any IDs more specific than genus until the 2/3 rule is met. It’ll add a bit of independence to the IDs.
3. No leader board. Number of IDs is part of your profile but there’s no easy way to see your rank.

I know gamification is part of what can make some sites popular and drives repeat visits but iNat has so much to offer. Maybe Seek is the app for games and ranks and rewards. iNat is its own reward.

[e-aus-kanada]

You can mute someone; they’ll still ID your stuff, but you apparently won’t get notifications. I haven’t used it, so I’m not sure how well it works. Edit your profile and then “manage your relationships”:

[jnstuart]

@tallastro

I know gamification is part of what can make some sites popular and drives repeat visits but iNat has so much to offer. Maybe Seek is the app for games and ranks and rewards. iNat is its own reward.

That’s been my concern about iNat from the beginning. If you make it too easy and too game-like, I think the quality control can start to suffer. I understand the rationale about making it inviting to a wide range of people, but I think it results in problems like the topic of this thread and brings out the competitive side of many users. Personally I’m not interested in being the top submitter or top identifier – I’m more concerned about occasionally contributing something useful biologically, even if it’s in smaller quantities.

There’s always been this tension between the iNat that is open to non-biologists with widely differing levels of experience and the iNat used by more research-oriented participants. I’m not sure if that will ever be resolved without major changes in the site and I’m not sure if such changes would be desirable.

[dianastuder]

or the leaderboard could only count actual IDs and ignore the subsequent agreements.

[lichenthusiast-jp]

Thank you @cmcheatle - I appreciate this info!

[obscurus]

I suspect the core of this issue is the fact that it’s so much easier to add an ID (even without the ‘Agree’ button) than it is to post an observation. Anyone can sit at a keyboard in the comfort of their home and rack up thousands of uninformed IDs in no time with little physical effort; some may just enjoy the game, others may feel as though they’re actually making a useful contribution. Adding an observation on the other hand involves going outside, taking a photo or sound recording, and then loading it. It’s much more work. Just think about the difference in time and effort involved in collecting and posting 100 observations compared to making 100 rapid-fire, uninformed IDs.

One of the ideas on an earlier thread that I think would help (not fix completely I admit, but help) is to have a probationary period for new users before they can add IDs. Let’s say you have to post 50 verifiable observations before you can add IDs for other people. There were arguments earlier that this would discourage new users. I’m not convinced it would discourage most who were serious about the site, but I think it would discourage many of the agree-bot users.

I don’t think this is quite as radical as it sounds. After all, we have a probationary period for adding projects and places—why not for IDs? It would just have to be made clear to new users that there were a few things they couldn’t do until they had submitted 50 verifiable obs, and that this was to try and improve data quality on the site (stressing perhaps that this includes protecting the quality of their own obs, as well as those of others). I don’t think many who were genuinely interested in iNat would turn away.

But, as so often here, we’re speculating when we need an evidence base. Again, we could adopt an experimental approach—put the ID-probation in place for a while and see if there actually was a drop-off in uptake. I’d be very much in favour of trying it. And I’m convinced there’d be far fewer fish-hooks this way than with a reputation system, and it would probably be much simpler to implement…

[whaichi]

@obscurus

Let’s say you have to post 50 verifiable observations before you can add IDs for other people.

While I imagine this would reduce the number of blind agreements from duress users, one of the top Coleoptera identifiers is borisb, who has 0 observations. I would hate to lose identifiers like him were requirements made too onerous.

[jimbee]

Personally being able to add IDs right away as a new user was one of the things I liked most about the site. Of course my approach to it went “Of course I know what this is” to “Oh wow, there are how many species of this??” The hands-on part is how I learn best so I’m not sure if I would have stuck around very long without it.

[clay_s]

@jimbee

How about renaming “Casual” to “Casual Grade” and changing it from grey to another color? It could just be a psychological thing, people want the nice shiny green RG badge of honor. I mean… If I’m being honest when I started on this site I felt the same way, that RG should be the goal. It wasn’t until I spent a significant amount of time here that I realized RG shouldn’t be and in many cases cannot be the goal.

That change might at least satisfy the part of the brain that hungers for badges and achievements?

That would be poor naming, “Casual Ungraded” would be more apt. People want them to be IDed I think, not some colored pixels. Not sure how that applies to “Needs ID” that most of these would be stuck at, not “Casual”.

[obscurus]

@whaichi

While I imagine this would reduce the number of blind agreements from duress users, one of the top Coleoptera identifiers is borisb, who has 0 observations. I would hate to lose identifiers like him were requirements made too onerous.

I don’t see this as an insurmountable problem. I’m sure the site admins could find a way to waive the 50 obs requirement for highly valued IDers.

More generally, we seem to be getting to the same point we did in the earlier threads—a lot of people have put thought into this and proposed possible solutions, but there’s been a problem with or an objection to all of them. I will say again that what we really need here is evidence, and there’s only one way to get that—change something and see what happens. If we’re not prepared to do that, let’s just accept that ID-spamming is part of life on iNat, and move on. We do seem to be going round in circles now. Maybe the forum’s done its dash on this topic and we should leave it to those who run the site to decide what to do (if anything).

[clay_s]

@obscurus

@whaichi

While I imagine this would reduce the number of blind agreements from duress users, one of the top Coleoptera identifiers is borisb, who has 0 observations. I would hate to lose identifiers like him were requirements made too onerous.

I don’t see this as an insurmountable problem. I’m sure the site admins could find a way to waive the 50 obs requirement for highly valued IDers.

More generally, we seem to be getting to the same point we did in the earlier threads—a lot of people have put thought into this and proposed possible solutions, but there’s been a problem with or an objection to all of them. I will say again that what we really need here is evidence, and there’s only one way to get that—change something and see what happens. If we’re not prepared to do that, let’s just accept that ID-spamming is part of life on iNat, and move on. We do seem to be going round in circles now. Maybe the forum’s done its dash on this topic and we should leave it to those who run the site to decide what to do (if anything).

How would they know his value if he was just starting? Or any future IDers? I wouldn’t recommend making special allowances for such proven people. See if they will jump through hoops to be allowed to ID here or whether they move on to other pastures. That will provide evidence on what impact the change has.

[blue_celery]

I understand that a stricter rule could prevent the id overconfirmation but let’s take into account that many obs are still in the need of an id because there is a lack of identifiers (both primary identifiers and confirmers).

[karen33317]

I like the leaderboard per each individual species. Hopefully it helps identify the real knowledgeable people for that species. Maybe eliminate the overall leaderboard, as that is probably what the “gamers” are pursuing.

[ksanderson]

Pretty new here so excuse me if I say something covered before.

I think if we added some information on the user to each record and/or just added a user exclusion line to the export query where you can use: number of observations, number of identifications, number of identifications/observations that were disputed, others?. This would let a researcher use those numbers to create a ratio or levels they are comfortable with and exclude users that do not fit your needs.

I agree with an earlier response about removing or change the identification leaderboard so it is less likely folks will just go on an ID blitz.

[glmory]

“There are some leaderboard fanatics who seem reasonably conscientious, and I don’t mind them. But I woke up today to find 51 observations with “IDs” that just confirmed earlier work”

The problem isn’t too many people going back and confirming thousands of research grade IDs, the problem is too few!

I have often gone through all observations of a species checking both research grade and needs ID. Every single time I have found incorrect research grade IDs in between the dozens of correct ones. Some species it is just one or two out of a hundred which are bad, some it is every fifth observation. It is bizarre to me how often people have complained about me making too many agreements with existing IDs. There cannot be too much review!

Also we definitely do not want to limit who can make IDs. There are quite a few experts on obscure taxa who only show up for a couple particularly interesting observations and otherwise do not contribute.

[kiwifergus]

Another point to consider… even if they are not an expert, by reviewing a large amount of observations, they soon can become one! We do this with children, showing them flashcards of “Tiger” over and over again… it is how we learn. By reviewing thousands of old observations, the overzealous identifier is literally just flashcarding themselves. When I encounter an over zealous identifier, if is usually someone going through and doing so for a single taxa at a time. When I investigate their identifications and activity (more out of curiosity than a desire to find fault) I typically find that they have a general area of interest, like birds. The evidence would suggest legitimate, positive use of iNat as a learning tool.

This of course does not work if you are just clicking Agree blindly…if they at least have to type in the name, or even just part of it (the 3+3 shortcut), they would be firming up the mental association of the image to that shortcut.

[dianastuder]

It would be good if we could, rely on the leaderboard for informed IDs.

I have learnt to click thru to their profile first and see if they look as if they know what they are talking about. Writing a Ph D on Bougainvillea - yes I will trust your IDs.

[gwark]

For what it’s worth, my experience with prolific identifiers (those that populate the leader boards), is that most of them are pretty helpful, and the site would benefit from more of them.

From time to time I do notice what I might consider overzealous identifying, but usually they aren’t up that high on id totals yet. Where I am at, the things that get attention (mostly birds, mammals and to a lesser extent vascular plants, marine algae and some intertidal organisms) are getting that attention primarily from prolific identifier. Most of them seem focused on particular taxonomic groups (perhaps within a fairly broad region), and a couple seem focused on intertidal invertebrates a little more generally.

I am also a prolific identifier, though I focus on an area rather than group. Part of my personal motivation for doing identifications is out of a sense of reciprocity for the help I’m getting from others identifying things I’ve submitted (I tend to aim for about twice as many IDs for others as I have observations). I also enjoy seeing what’s out there for me to look for in the future, and learning more from what other are seeing.

My approach is to look at everything in a specific geographical area, centered on Southeast Alaska (all of my observations are from this region, with the vast majority from the Sitka area) - but spanning coastal areas from Haida Gwaii to the Aleutian Islands without regard to taxa.

Each day I try to look at every new observation (I have bookmarked the identify page for the patchwork of locations that make up this broader area), and identify everything I can (being more conservative the further away something is from my core area).

As a result, I’ve accumulated many IDs and have become top identifier in multiple different taxonomic groups, not because of any particular taxonomic expertise on my part, but because I’ve learned what occurs in my area, and if it’s something that’s pretty common, I end up with a lot of IDs, even though I don’t actually know how to identify the organism from other similar looking ones that might occur elsewhere (in many cases, I don’t even know if there might be similar species elsewhere).

Incidentally, I do get tagged from time to time because I’m on the leader board for the species, and if it’s outside my area (it usually is) I politely decline to offer an ID because I can’t really confirm outside my geographic area.

I do benefit greatly from those folks who focus on a narrow slice of taxonomy with little or no geographic constraint, and contribute to observations in the area I cover. Their IDs help me better understand what occurs in my area - and there are several I can think of who I’ve come to appreciate for their knowledge and help.

[JayAvery]

I agree with you about the Agree button! :D

I notice that new users who post an observation with no ID often Agree with whatever ID someone else makes. And I think it probably just comes from a slight misinterpretation of what the Agree button actually means. It’s a friendly green button and it’s right there on the corner of the first response a stranger has made to your first post on a new site - of course you want to click it and say thanks! More evidence of this use of the button is when an observation gets two or more IDs and then the observer has clearly clicked Agree on each identification in turn, so they make and withdraw 2+ IDs in quick succession (sometimes even of the same taxon).

I agree with you too about the solution. I think at minimum, some changes are needed to the way new users are taught how to use the site and what Agree means. But I also think the best solution would be to scrap the Agree button completely - like you said it’s not that hard to start typing the name in order to add the ID. I also think this solution will never be accepted though :D The Agree button is really convenient and I think there would be so much pushback it will never actually be removed.

[lvdt]

An alternative (and maybe easier to accept for existing users) solution would be to rename it to something less “friendly”, but more descriptive of its function, such as Confirm.