Wild idea on community ID: let user pick the model they want to use

Has anyone thought above solving the community problem by having different formulas and allowing the user to select which one they want to use? By “which one to use,” it would mean for any observation, map, seasonality, Identify session, etc. they display, the data would be filtered according to the model they choose. For those that like the current model, things work the same as usual. Those that prefer the model that gives experts heavier weights will get what they want. That way everybody gets what they want. Sort of. I imagine the problem would be that many of those that pick the model that give experts heavier weight would stop trying to confirm IDs once they get a consensus under their formula, thereby worsening the lack of good consensus ID for those opting for the current model.

1 Like

I can’t get past an image in my mind of a football match where some choose American Football rules, and others choose Gaelic Football rules… Throw in Rugby League and Rugby Union as bonus choices, and it would be a pretty weird game!

I think the most crucial aspect of any system we use is that it be consistant.

14 Likes

this gets messy really fast. first, does anyone even have a well-defined alternate observation taxon model that has substantial support as a potential alternative model? second, once you have such a model, you have to create all the supporting setups and processes for it. for example, you if you want to define some sort of expert-weighted model, you have to have ways for defining who is an expert at the very least. third, you’ll need to implement and support it. that’s no small task. (you’re basically asking for two parallel views of the same underlying data. so it’s at least double the work.)

@dan_johnson, if you can raise several million US dollars to pursue such an effort, i think that could get some discussion going on this idea, but i bet you’d realistically need more like eight figures even possibly tip the scales in favor of moving in this direction.

in the meantime, if you’re just interested in, say, cicadas or crayfish, you can always pull the data out from the system and analyze and visualize it however you want. you can apply whatever kind of observation taxon model you want to use. if you want your own IDs to override anyone else’s IDs, you can already do this. if you have a core group of experts whose IDs you want to give double weight to, then you can already do this.

6 Likes

Considering all the discussions on alternative model, there must be a lot of interest.

Do you really think it would cost that much? If so, that would be a reason not to pursue. Are you just speculating or do you know the software well enough to assess this?

I’ve been a software engineer for 30 years and it’s not at all clear to me that this would double the cost. For stuff I work with it could be not particularly difficult. I work with different data structures, so perhaps the data structures of inaturalist are stored with the community ID build in. If that the case it would be difficult. Perhaps certain data structures would need to be stored in two copies, one for each model. That would be prohibitive.

because observation taxon is such a fundamental thing in the system, there are very few parts of the system that would not be affected by what you’re suggesting. what you’re proposing here is a major, major change in the context of iNaturalist. the dollar figures that i’ve mentioned are back-of-the-envelope kinds of figures, of course, especially since we haven’t even defined what the alternative observation taxon model would look like, but i guarantee you that what you’re proposing would not just be a matter of someone knocking something out in their free time over a few weeks. it’s not just a technical issue either. there are lots of stakeholders and interfacing systems that would need to be considered.

It’s fine to propose ideas, but maybe let’s finish fleshing out at least one of the many that have been proposed (for example, https://forum.inaturalist.org/t/add-ident-user-taxon-id-and-unident-user-taxon-id-url-options/14885/7) before we move on to another…

2 Likes

some sort of crowd-consensus voting system?

Here’s something to consider, the next time you upload an observation from a lesser known group, look at who the top identifiers are (the people most likely to ID your observation), if you read their profiles, they often are experts in their fields. Look at the top identifier for bees, ambush bugs, robber flies, lightning bugs, ect. So from what I have seen, when you upload something not well known, the person who ID’s it, often has a degree in that field. And if you want to be a little more sure, than don’t put a species level ID yourself, and don’t agree with the first ID’er. Another option is, cultivate a relationship with an expert in field you are interested in, and tag them with tricky observations.
Edit: I would also add that there are already options for getting experts to ID observations, odonatacentral.org, beespotter.org, ect. and to be honest, I don’t always agree with the expert ID, but on expert platforms that I have seen, there isn’t a lot of back-and-forth explanation of ID’s, on iNat I have frequently asked and been asked for clarifications and take the time to explain satisfactorily.

4 Likes

This seems pretty obviously divisive. It would effectively create two different communities, and would risk inviting users to take sides in various ways - e.g. I prefer Formula A, so I’ll also prefer IDing observations by other users that prefer Formula A.

1 Like

just to clarify, i don’t think dan_johnson is saying that the observer’s preference drives the algorithm that is used on a particular observation. i think he’s saying that each observation can be viewed as if the taxon is driven by either algorithm, and each viewing user can decide which algorithm to apply to all the observations they view. so for a given viewer of the data, i don’t think observation A would be different from observation B even if their observers preferred different algorithms.

still, having multiple views of the data does fragment the community, because some would see things in an entirely different way. so that’s definitely something that needs to be considered if going with this approach.

1 Like

Different formulae with different parameters and different results requiring additional database fields for every option. More bulk, more moving parts, more work for staff and curators.

For me it comes down to “Does this contribute to the primary iNat mandate of promoting understanding of biodiversity?” I don’t think it does and it probably gets in the way.

5 Likes

And nor am I. There will be two views on the same data, which will subtly influence the way users interact with the interface and each other. In time, this may lead to the creation of two different cultures within the same community, which could become divisive.

1 Like

I think all the photos on iNaturalist should be identified to the same standard, whatever that’s going to be.

4 Likes

Blockquote
For me it comes down to “Does this contribute to the primary iNat mandate of promoting understanding of biodiversity?”

In my opinion, yes, because if there is a second community ID model that optimizes identification accuracy and numbers, those that want more accurate maps, seasonalities, etc. will probably be able to get substantially more accurate ones. More accuracy means better understanding of biodiversity.

1 Like

Keep in mind here, I am not proposing a new community ID model. There’s already a huge discussion on empowering experts. Any alternative community ID model that gives expert IDs higher weights is not the point of this discussion. I think developing a model like that could be quite costly, but could perhaps be worth the cost. In the context of that discussion, however, the main idea is that those that are considering the different model might consider allowing both. The user in there settings would just set the model they prefer using and then go about there business and at any time switch to the other model.

1 Like

Hold on - why would one model be more accurate than the other? Surely both views would have to be equally valid, otherwise everyone will just switch to the most accurate one (assuming there’s an objective way to measure that).

Just for sake of argument I’ll give two simplicist models: 1) the current community model, 2) IDers with greater that 1000 IDs in a taxon get a weight of 3 and other people get a weight of 1 (so the former’s ID counts as 3 IDs). Which model will have more accurate community IDs? The 2nd one almost certainly would.

I’m not at all suggesting the above option 2. I think a good one would be much more sophisticated.

Your proposal is starting to look like an attempt to introduce a reputation system through the back door.

My impression is that there has not been any serious consideration of a reputation system for iNat. Yes, it gets brought up by users in the forum, but I think iNat staff have resoundingly rejected those type of approaches when they have weighed in.

5 Likes

see https://forum.inaturalist.org/t/implement-an-ispot-style-reputation-system/6998/82

5 Likes