i feel like even in true field work where you collect samples, that bar is higher than anyone gets - utmost certainty? Maybe with genetics but even that…
I sometimes include a google streetview image link to trees, which in my opinion can in certain circumstances (case by case, e.g. which species, location, etc.) be sufficient to ID to species and frequently to genus at least. In the future, I would be interested in having such publicly available images be usable as evidence e.g. for dead/gone trees that can no longer be photographed in person, or for very remote locations where it would be impractical to travel to
1 Like
i use google street view all the time but when this came up a long time ago (because of the defunct Darwin for a Day project) i think the response was since GSV images change and since they can’t be copied into iNat for copyright issues etc they don’t ‘count’ as research grade.
There’s a huge world of data that can be lifted off of Google Street View.
2 Likes
in the context of iNaturalist, i think evidence is anything that can be used to make an identification. what constitutes good evidence vs bad evidence is another thing, but i think in the context of iNaturalist, it may not matter.
this discussion reminds me a lot of debates over the merits of proof of work vs proof of stake systems in the blockchain world. in that world, you’re trying to take the need to trust a single central party that controls the system and shifting that trust to the system, and specifically to distribute it to as many stakeholders in the system as possible.
but how do you trust that all those anonymous actors out there have an incentive to act in the common interest? right now, the two main models for proving that you’re going to act in the common interest require you to either do some sort of measurable amount of work (why would you go through all the trouble if you’re just going to cause chaos?), or to put something at stake (why would you act badly if you’re going to lose what you’ve put at stake?).
iNaturalist does sort of seem to be trying to shift trust to the system as well, and their implementation of it to me looks a lot like a proof of work system, albeit with a very low work threshold (click a button to ID), though there are hints of a proof of stake system where just social approval is at stake (and that will probably exist in any social system).
the reputation system that some folks have advocated would push things toward more of a proof of stake system, where “reputation” is at stake. i won’t go into all the pros and cons of that approach or a hybrid approach, but i will say that my preference, and i think what is better in the context of iNaturalist is to stick with a proof of work system.
so then how to you improve things if you’re going to stick with a proof of work system? well, you increase the amount of work that needs to be done before an ID can be made. theoretically, that work can be arbitrary, like having to solve a random math problem. but you can also make it something that helps with that particular ID or that helps to improve some other aspect of the system. the tricky thing is that you don’t want to make the work so burdensome that it disincentivizes people to make IDs.
what does that mean? well, regarding the disincentive, i think maybe you could minimize that by requiring the extra work only in certain circumstances. for example, maybe there could be some algorithm that compares your maverick IDs vs your leading IDs that became RG or something like that and uses that to weight a random chance that you’ll be required to do the extra work before you’re allowed to ID. (you could implement a hybrid system here by exempting certain people from the extra work based on some sort of, say, curator or expert status or something like that.)
regarding extra work to help your IDs, i’ve talked about identification checklists that i think might help lay the foundation for that (see: https://forum.inaturalist.org/t/add-often-confused-with-warnings/1269).
and regarding extra work to help the system in general, there have been different ideas for custom CAPTCHAs (https://forum.inaturalist.org/t/limit-new-users-ability-to-send-messages/469) that could lay the foundation for that.
maybe you can even combine the concept of the checklist and the CAPTCHA to help the computer vision understand, say, which quadrant(s) of a photo contains a bird wing, or something like that.
sorry i didn’t go into a lot of detail here, but i wanted to put the concepts out there, in case that might open up a useful direction for the discussion. hopefully it’s not just a bad tangent.
3 Likes
So far for iNat i think photographs of the organism provide the best piece of evidence. Secondary evidence such as tracks, nests, calls etc, are also acceptable to me since something had to be making them, and I’m sure there are many experts here that can provide an ID based on that kind of information.
I personally am not so sure about illustrations, the biggest issue is the risk of human bias (and also the artistic skills of the observer). I mean if the observer is out in the field sketching a flower, I think its fine, but if he/she is drawing a butterfly, that butterfly would have likely flown away long before the illustration is finished. Thus the observer will have to complete his/her sketch essentially from memory, which can lead to a potential bias or false memories (whatever they are called) on how that butterfly looked. Maybe he/she drew an ocellus in the wrong place, or a tail thats projecting out from the wrong vein.
For “an attached description or comment by the observer describing a sound or feature not depicted in the photo”, that may work but of course is dependent on human interpretation and how articulate that description is.
I wonder if inat can allow like a 5-sec max video evidence (though thats likely gonna take up a lot more database space)…
1 Like
some have proposed requiring more IDs - 3 instead of two, or however that works out in the community ID algorithm. I don’t like that idea but I do like the idea of Identify giving you an occasional research grade identification… say, 1 in 20 is already research grade, the rest are ‘needs id’. And it could keep track of how many disagreements there are, and with taxa with more disagreements, it could show those more. For instance if everyone is mixing red and black oak, it will be more likely to show you a research grade black oak than a research grade white oak, if no one is mixing white oaks up with anything else.
Could be hard to code but would work well i think.
1 Like
just to clarify, i’m not talking about requiring more IDs to get to a community ID. i’m talking about requiring more work before the system will allow one person to make one ID, on occasion, based on some sort of chance algorithm – sort of like a random traffic stop that becomes more likely if, say, your headlights are out.
here’s an example of what i have in my head:
in my area, we have Plestiodon fasciatus and Plestiodon laticeps, which look very similar as juveniles. a lot of people see blue tail and lines on back, and automatically assume 5-lined skink. if we implemented the “identification checklist” that i’ve talked about, someone could add, say, “4 labial scales between front of eye and nostril (vs 5 scales in P. laticeps)” to the P. fasciatus taxon ID checklist. that checklist could in most cases be used for reference purposes or maybe optionally as part of ID.
but if the system identified P. fasciatus as a species where there were a relatively high number of community ID changes from species level back to genus, and if it identified Bob as, say, having a lot of cases where his IDs tend have lots of disagreements, then maybe the system stops Bob while he’s making a P. fasciatus ID, and forces him to go through the checklist: “To help the system make better IDs, please take a moment to go through this ID checklist.”
maybe Bob gives up right then because he decides that he doesn’t actually know skinks that well, and he’s not going to go through the effort just to prove that he doesn’t know skinks that well.
or maybe he goes forward. when he gets to the “4 labial scales” item, the system takes the photo of the skink, breaks it up into several sections (like some of implementations of CAPTCHA) and asks him to click on the sections that show the 4 labial scales. maybe Bob pauses for a second and decides that he really can’t see the scales and stops his identification, and clicks a button to say he’s unsure about his ID. if he makes a higher-level genus ID at that point, and others later agree, then the computer vision maybe learns that 4 labial scales is important to making an ID.
maybe Bob decides to go through the entire checklist because he really wants to make an ID. in that case, maybe he made a bad ID, but he was sure enough in his ID that he did the extra work to make it, and at the very least the extra work slowed him down. and if it turns out that others with good ID track records disagree, that increases his chances of getting stopped again for his next ID and slows him down in the future, limiting his potential for damage. or if others with good track records agree with his ID, then maybe the computer vision can then say, ok, those sections of the photo that Bob pointed out probably contain labial scales (or something like that).
regarding your other point about “1 in 20…”, I don’t know that I would offer up random research grade observations to be identified by random people, at least not where it would actually have the potential to change the community ID. (i think it would be ok just to gather data about where there’s potential for bad IDs though.)
all of these things are big asks of both coders (to make the system changes) and the community (to add/curate all the extra metadata, learn the new processes, and endure the occasional “traffic stops”), but i think if you’re really trying to create a system that can systematically capture when people make a greater effort to ID things, potentially educate and stop bad IDs from happening, and, by the way, is backward compatible with existing community IDs, that’s the path that you want to head down, i think. (you’d probably break it up into several phases for sure, though.)
1 Like
hmm. well, i don’t see how getting a third opinion by a ‘random person’ is worse than just having the one id by a different ‘random person’. And to me if checklists and quizzes were added when trying to do IDs, i’d probably do a lot less IDs. That sounds annoying. Maybe the occasioal informational popup “did you know Toxicodendron rydbergii also occurs in this area? It is difficult to distinguish from Toxicodendron radicans if you only have leaves” type thing with a ‘dont show me this again’ checkbox. For observers as well as IDers.
The bottom line is, there are some taxa with high error rates, but overall the error rate of RG is pretty low at least in the areas and taxa i look at. The only real way to help now is getting more people to look. Maybe the system could also show people things they have had success with. In the long term i know the inat devs are looking at an intrinsic reputation system to weight IDs and that might help a lot with this too.
2 Likes
the difference is that in the latter situation, that rando was going to make the ID anyway. i wouldn’t want to invite a rando to make an id on a research grade, potentially messing up the community ID. (based on what i see, once there’s a consensus, it’s usually only people who have more experience who continue to add IDs.)
anyway, i’ve strayed too far from the original topic at this point. apologies to the discussion originator.
1 Like
similar to how others use other resources like google street view, often, i’ll reference other observations with better photos as evidence for observations with poor quality photos, if i can logically tie them together. for example, this observation of a pileated woodpecker (https://www.inaturalist.org/observations/21242065) with a grainy photo taken on a phone from a long distance probably was never going to achieve a research grade on its own, but i bumped it up to research grade based on another observation that was made roughly around the same time in the same location.
3 Likes
I’m pretty conservative about what counts as “evidence” personally - to me, unless it is verifiable by an outsider looking at the post, it shouldn’t be counted.
I’ve been on field trips with highly-educated expert botanists who mis-identified a common plant in a moment of inattention - it happens to the best of us. However, if they were to post that and people just agreed based on their reputation, that would be a significant error in the data.
And illustrations, unless the artist is a trained botanical illustrator, will be iffy at best. They may miss key features, be unconsciously biased, or simply lack the skill to record it in any identifiable manner.
8 Likes
i think we’re saying the same thing here at the end of the day. the algorithm that decides when to do a “traffic stop” would definitely need to be tuned properly to avoid disincentivizing IDs, and how much work you ask an identifier to do during a “traffic stop” – whether just clicking ok to an extra pop-up or something more challenging – would definitely need to be tuned as well to avoid disincentivizing IDs.
3 Likes
This is an interesting thread. I just wanted to push back against the view that observations without evidence should be discouraged from iNat because they are unlikely to be of use for research. I disagree, completely, and I say that with my university researcher hat on.
To do research with biodiversity data, it is ideal to estimate the probability that an observation ID is correct. With museum and herbarium collection data, it is usually assumed that mis-identifications are unlikely and IDs can be assumed to be correct. That’s not the case with iNat data, whether or not an observation is “research grade”. (We probably shouldn’t be ignoring the mis-identifiction probability in collections either.)
The probability of an iNat ID being correct can be estimated using a combination of the proportion of times particular observers and particular identifiers have misidentified a particular taxon. Some taxa are also inherently easier to ID than others, which can be quantified with iNat data.
If there’s a “research grade” observation with a photo, it could still have the wrong ID. That probability could be calculated from the frequency with which this species has been misidentified in other iNat observations, and the track record of the observer and identifiers at identifying this species.
If there’s no photo with an observation, the ID can still have a high probability of being correct. That’s when that observer has an excellent track record identifying that species on iNat, especially when that species is infrequently mis-identified generally. These probabilities can be estimated with iNat data.
In other words, for both “research grade” observation and observations without evidence, the probability that the ID is correct can be estimated using iNat data. In both cases, observations with a higher probability of being correctly identified are more useful.
We shouldn’t be thinking of each iNat observation sitting alone and independent. For research, it’s just as important to know the context of the demonstrated accuracy of the observers and identifiers.
My view is that we should be encouraging everyone to get outside and make lots of observations of wild species to document the natural world and how it’s changing. That’s with and without photos.
If a particular species is “your thing”, and you have demonstrated on iNat that you are very good at identifying that species, it is counterproductive for us to say that iNat only wants your observations with good photos. I’m worried by the thought that we encourage these users to keep all of their other observations on a spreadsheet on their computer because we tell them that they’re of no use to others. That would be a massive lost opportunity to better document nature.
(If our objection to observations without evidence is just that they clutter the site and get in the way, then that’s a UI issue, not a justification to discourage these observations. iNat is already very good at keeping these observations out of view from default searches.)
17 Likes
I agree! Well said!
2 Likes
Not just well said, but a very good explanation of the “wisdom of the crowd” aspect at play with iNat. I’ve always considered that even if the data isn’t 100% accurate, the general picture that the data creates is very useful, if only in identifying areas worth more rigorous scrutiny. I think because we come from a position of dealing with the data on a piece by piece basis, we forget what statistical analysis can do with “big data”
8 Likes
i do add a fair bit of photoless data, for instance i am slowly adding some old plot data for surveys i did years ago in the Santa Monica Mountains. I figure people can judge for themselves because there’s plenty to go by in terms of my other observations and comments on the site… and anyhow the data has already been used, it was just never put in GBIF. I have other side projects and things too. ata is data, building the community and connecting with nature is important, but so is data. We all know we are in a horrible biodiversity crisis and data is one of the most important things that may allow us to mitigate it.
4 Likes
(Apparently I’ve had a reply in draft mode for most of a month - yikes! I’ll preface that the following comes from the perspective of entomology, where sometimes the number of species in a genus exceeds the number of total species an individual may know about.)
I think the main question is whether the evidence presented represents a legitimate observation by the submitter, not just what types of evidences as permissible. I think there’s also the thought of how specific the evidence actually can support. For consideration:
Does a drawing of an arthropod have 8 legs, wings, and apparent feathers represent a species record? Almost certainly not. It directly contradicts any existing species.
Does a drawing that looks like an amoeba represent an observation of an animal? I’d say probably not. With species records, this really doesn’t provide support of seeing anything even though it may have been intended to represent an observation.
Does a drawing with at least a vague semblance to a described organism represent a species record? I’d say probably, though it may be a fair point to ask if this represents a sketch of a wild-observed species as opposed to more general artwork. It may not necessarily support a species-level ID, though.
As someone quite familiar with paper wasps (incidentally, one of the taxa with rather high rates of error), does my description of a wasp represent a valid species record? I’d agree with Charlie and the other Jon that it probably should be considered as such. Heck, I’m often asking questions about traits to supplement observations, and it’s that description that validates a species. I have at least one observation where I managed to see the wasp up close rather nicely (to the point of a usable description) but wasn’t able to get a decent photo as I had to chase after it when it flew away. Now, at the same time, I do often get a bit perturbed when scientific authors make claims of observing some particular species with no form of (adequate) description or photography (how was it separated from cryptic species B?), so I think there should be some weight to validate that particular species beyond reputation.
Would DNA sampling info validate an observation? If the DNA came from an observed, wild specimen, I would probably say that this is the single best evidence possible, provided there’s decent barcoding for its group. This goes down to what actually defines the physical differences we may observe between species.
Does an observation lacking anything but species ID, location, and time represent a species record? I think this really has to depend on the identifier’s knowledge of the species in addition to the range (whether the species is already known to occur in that area). I’ll offer a counterpoint to Jon’s thoughts in noting instances of professional misidentification of museum specimens.
I think there’s also something to be said about quality of evidence. If an observation’s photo (and other evidence and observer background, for that matter) lacks sufficient detail such that it can’t be proven right or wrong, then there may be a need to go case-by-case as to whether it’s appropriate evidence. Cryptic species immediately come to mind (considering Jon’s notes on the probability of a particular ID to be correct, these are groups with very low rates of accuracy where the user base tends to be unaware of all but a single species). Then, of course, there are species that are painfully difficult to misidentify.
5 Likes
I used to be on the records committee for the Iowa Ornithologists’ Union. We evaluated observations of rare birds. A problem: one guy knew his birds so well he could ALWAYS write a convincing description. But one day when I was with him, I swear he didn’t actually see species A, which he described well.
I think iNaturalist shouldn’t try to be all things. Stick to evidence we can evaluate directly – photos, recordings, sonograms. Let other organizations deal with written descriptions.
8 Likes
A post was split to a new topic: Encouraging a sense of scale in photos
I came across this conversation and I’m curious what people think: https://www.inaturalist.org/observations/35150745
Apparently the virus is necessarily present, but there’s no direct evidence of it. In my observation the lady beetle would not be behaving the way it was without the activity of the virus, so I think that’s direct evidence of the virus. But if every observation of the wasp species is duplicated for the virus, is that even useful information?
I’m guessing there a number of species of bacteria and virus that are present in every human on the planet. Theoretically you could make a duplicate observation for Escherichia coli for every mammal and bird observation. Is this different from that in any way? Would that be a valid observation?
3 Likes