Ha ha point taken!
The really key issue has already been said but it’s this: profs/teachers need to curate their students’ observations very carefully. If they’re assigning iNat, they need to grade it like an assignment. I’m not sure how to communicate how critical this is to teachers, lots of work has already been done on this front.
Could there be some sort of positive reinforcement or reward system for student projects? Like we could assign them certain badges based on quality of observations or something? Just putting it out there.
We’re doing some testing for this now. I think it will create a way to escape a contemporaneous flood of poor quality observations for those who don’t want to see them, and will conversely allow someone to focus on those observations if they want to be part of a welcome wagon.
This approach is actually promoted in our Teacher’s Guide due to privacy laws and to allow teachers the ability to curate observations.
I think the single biggest issue with iNat use in classrooms is that it is often used by a teacher who has not had a chance to understand iNat - what it’s for, what it’s good at, what it’s not good at, and most importantly, that every observation is not uploaded into a vacuum, but to a community that will be affected by it and may feel responsible for it. I know teachers (at least in America and I’m sure in many other places) don’t have much free time on their hands, but ways to make sure teachers understand iNat is, I think, a key here.
We link to the Teacher’s Guide in our introductory email, but I imagine most people don’t notice it. It can definitely be improved as well. I think some sort of threshold one needs to pass before being able to create a project, eg 25 RG observations or something, could prevent some inexperienced teachers from using iNat in their classroom, since they usually start projects. Then they would (hopefully) know whether iNat is even a good fit for what they want to do.
And I think a fundemental point that is being missed is that problem users (might be teachers, students etc) are not going to use ‘student’ accounts any more than they’re going to try and understand what iNat is all about before posting observations … unless they can see that it is an advantage to choose this sort of account - as I said ask ‘them’ what they need.
Or unless that’s the default unless they choose otherwise. Or unless we put them there.
I’ve had a bit of exposure to different projects used for students, and I would have to say that the quality of data absolutely correlates to the educators, not just the students. Many of the problems come from wholly unrealistic expectations of instructors and, frankly, poor instruction. These issues are fairly common whether the group of students are in grade school or university.
• expectations to submit large amounts of observations
• expectations to submit large amounts of IDs
• expectations to ID to species
• expectations by instructors that CV is always accurate
• no training in photographing specimens
• no training in identification
• no training on the existence of distribution ranges
• instructors do not help curate student observations
• instructors don’t consider student maturity levels
• instructors have never used iNat themselves
• users don’t respond to @-mentions due to only using the app
Many of the problematic requirements are in rather stark contrast to how my undergrad entomology course was handled. We had really fairly low requirements, all things considered (though we were just handling things in-house, not through iNat or similar platform).
• observation numbers were low (50 preserved specimens, 25 photos - not in the 100s)
• IDs were only required at the Order and Family levels (and we still had a pain with beetles)
• IDs were mostly done in-lab where we could get assistance, not just at home
I think really a decent first step would be to more formally require that teachers demonstrate at least passing familiarity with the Teacher’s Guide since the majority of issues are covered there. Not too long ago, there was a problematic course project in Ecuador where one of the main issues was that the instructor based grades on the student with the most submissions and where the instructor had been away through most of the project. These are red flags that really should have been nipped in the bud on the early side when it came to creating the project. I’d also suggest that these issues detract from students’ experience and learning.
But, of course, student assignments involve students, and students don’t always follow instructions (mine certainly have fallen under that category). There are also some additional issues for non-university students in general that will differ from university students. As a whole, student IDs are almost always restricted to course projects, so some sort of “student account” could sort of be like training wheels. However, getting them to actually be used could be difficult unless that were also a part of the requirements for teachers to create course projects.
Maybe instead of separate student accounts, it might be possible to consider similar functionality to Discourse. There are tiers of account, where the initial tiers are a bit restricted until they complete the interactive tutorial. Since a lot of problems involve “newbie” accounts agreeing with each other, maybe some way to prevent research grade status with only IDs from such accounts? It needn’t require crazy requirements to tier up, just something that users under duress probably wouldn’t end up brute-forcing by accident.
Of the two, I would opt for making IDs from new accounts not count towards the community ID, rather than adding yet one more thing for site curators to do.
@jonathan142 Well said and thought out Jonathan! Thank you for a very helpful perspective.
I think educators / teachers should pair up with local iNat power users, and be mentored in how to run their assignments/projects. I would volunteer!
I’m a fan of the idea of probationary accounts for all new users:
- Observations and IDs made under probationary period are flagged as “under probation”
- Probationary IDs don’t count to CID
- Optional “Mentor Account”, which can log in and make limited changes in an account (ie not able to view personal/direct messages, but can fix dates, pin locations, withdraw and apply new IDs, etc)
- Mentor account or a curator/staff can release the probationary account early, for keen and capable students or for experts/specialists signing up. Option to release all existing IDs/Obs from probation when releasing an account early.
- Individual probationary IDs or observations can be released, one at a time, once mentor/curator/staff have checked them.
- Other users can have an account setting to either include or exclude seeing probationary obs/ids
I would imagine that for a lot of users that show responsibility and capability from the outset, it would just be a simple release account and obs/ids, and by the time the probationary period is up (after which IDs and Obs made are normal), most duress users would have lost interest and/or requirement to be using the site.
Another possibility is to have teachers give US the ability to vote on whether the content/interaction from the student warrants a pass or a fail!
Just initial reaction – I’m thinking this would be a bit heavy-handed.
What would you think of having the same probationary/mentor system, but have it be assignable by a curator, instead of be the default from the outset?
That way curators could visit the “recent users” link and look for a bunch of new sign-ups that appear to be from the same school, and give them probationary status. Or at least respond to flags and other user input with that additional option. It wouldn’t be perfect, but maybe not so onerous or off-putting to new non-duress users.
I would definitely support initial probationary status for some kind of special student or class account if that can be implemented. Then we just need to brainstorm the best ways to funnel student/class users into that account pipeline.
And I love your idea of being able to weigh in on student grades! (Would probably want to see the teacher’s syllabus first, though, to know what the initial expectations were…)
I’m not a huge fan of “probationary accounts.” What made me bold enough to try iNaturalist for the first time was an NPR article that explained what the site was for and how user friendly it was, and then, when I did try it, the ease of use and friendly tone established on the site made me come back. I never felt “stupid” for not knowing how to do something or that I was wasting any expert’s time by posting. People were encouraging. I realize students forced to use the site gum up the works (even as a teacher I find the blurry pictures puzzling–surely, someone gave better instructions about photos . . .), but I’d hate to put all new users on “probationary status” to weed out unhappy students. It just seems less friendly and open–it ups the “risk” for a new contributor–even if it’s only a small extra step because it presents itself as a test. If your background is not scientific and you don’t have experience photographing organisms, you’re going to make mistakes. I make lots of mistakes–and I’ve been here since 2016. I plan to make a lot more, but I would hate if someone zinged me for them because I’m truly making an effort to do something I love.
I wonder: Is it possible to have a teacher register for a school account and could observations on the school account have a “publish button”? (Rather like online grade books–you input grades but no one sees them until you hit publish.) The teacher could review the observations that have been uploaded and then “publish” those of acceptable quality to the rest of iNat community? Or is this too complicated? Anyway, just a thought.
That was your first experience of the site. If all accounts are probationary at the start, then that will be their first experience of the site, and that first experience won’t be hugely different from what it is now, just that CID won’t be affected by your IDs and some users might have opted to not see your IDs and Obs… how would you even know the latter was the case? And for the CID, unless you knew it used to be different, how would you even know that was the case?
For a large number of new users now, their first experience of the site is getting lambasted by the hyper annoyed regulars for posting blurry photos and/or having a laugh by IDing their mates as baboons. Hardly a friendly welcome! At least with probationary accounts and them being filtered out by those that are bothered by them, it is just the cool-headed and patient regulars who get to interact with them! I’m just as concerned about losing the expertise of our regulars that are starting to leave because of the nuisance factor the duress users are presenting.
It is far more welcoming to new users to be granting them more access as they continue to engage, rather than starting at a level that then gets chopped back just because they are part of a school group. That would have the effect of detering the few amongst them that might go on to become regular and power users. Just look at this forum, and how it “opens up” as you use it more! I didn’t find it off putting at all… it just encouraged me to be more active!
Yes, of course, you’re absolutely right–it was my first experience using the site–I can only speak from that experience. I’m just saying I don’t see why the “hoops” have to apply to all new users if the problem is students. I also don’t see why teachers can’t oversee school observations and make choices about them. If the problem is overwhelming people with awful observations, then the first line of defense should be giving the teacher the right to publish or not publish these photos–if possible. Students are used to teachers making decisions about their work and calling them on bad behavior. If a teacher says it can’t be published and explains why, the student learns just as much as if he/she is denied full access via the site curators. The difference is that the feedback comes from the person who is trying to teach them and knows them personally. Other new users don’t get denied full access, just as the rest of us were not denied full access when we started.
Honestly, I haven’t seen that many regulars chewing out kids. As a teacher, I would quickly decide not to use the site with a class if I thought users were likely to berate kids, so if that is happening, then I’m guessing we’re going to see teachers choosing not to use the site as a teaching tool. (I teach literature, so this isn’t an issue for me, but all teachers care about student online learning experiences. They don’t send kids where kids aren’t wanted.)
I agree we don’t want experts leaving. Are they?
At any rate, I was just offering an idea because I have some reservations about making everyone who signs on probationary and limiting access when people seem especially concerned about students. However, if all new people, from students to retirees to experts in their fields, are subject to the same scrutiny, then it is fair at least. I still can’t say I think it’s great idea, but then I hardly matter in the grand scheme of things. Thank you for sharing your perspective with me. It is interesting to hear other points of view.
many good suggestions above. So, let me just toss out my takeaways from the most recent City Nature Challenge in Penang … This was from all accounts I can discern a total disaster.
Problem: many of these brand new accounts posted 100+ observations that single day, now while I take pride in a good species century myself, I also typically get 95% of that century to RG within a few days, by taking good photos, and knowing what kinds of features lead to solid IDs in many classes of organisms. Expecting 100+ good observations of wild things from a new user is just not realistic.
Solution: limit uploads to 10-20 per day (they can take a week to upload if they choose
Solution: checksum limit photo resubmission, one potted cactus was loaded by one user 30 times, with 30 different IDs (one might even have been right?)
Problem: I think users were encouraged to try to get to the “top” of the ID standings. I saw several users that made several hunderd (400+) IDs that one day. Of course all were AI/CV assisted IDs, and (since the computer hasn’t seen anything in Penang before), they were all wrong
Solution: do not allow CNC challenges to even know their ID count no counting of IDs (that they can see), or Do not allow new users to place IDs deeper than order (yes, they even get Class IDs wrong (Dicots as Pinales, Pinales as ferns, etc)
Problem: After uploading those 100 obs in a day, or making those 400 IDs in a day, they have vanished from the site, now those in-the-wrong-class observations need to get the right class (bumps them to Tracheophyta or Animalia), so then the next 2 IDers have to be willing to troll through that huge set) and it takes 3 right votes to fix. It has been 3 weeks, of those 1500 IDs I’ve made on Penang, about 10 have had an ID of_any_type added after I made an ID, so unless I @ someone, they are dead observations.
Solution: See Two solutions above
In short, we gathered here are all in the business of sorting things into bins, that is taxonomy. The Penang observations amount to a very small quantity of gems needing to be sorted out of huge heap of trash (60,000) that we have allowed to be dumped upon us.
Solution: Try some: https://www.inaturalist.org/observations/identify?quality_grade=needs_id%2Cresearch&project_id=penang-intersecondary-school-city-nature-challenge-2019&captive=false
and yes, even the butterflies on Penang, half are in a butterfly FARM !
Ugh. Thanks for the links. Did my part and got a page of Unknowns into the right bins at least. (Unfollowing each one along the way…)
Good suggestions all, hope some are feasible.
Fair points. So far, we on this forum are a pretty small and self-selected group, however, so I don’t know that we can yet project our experiences to the majority of iNat users. But over time, maybe so…
FWIW @lincolndurey, @carrieseltzer and I spoke with the organizers of that event and I wrote a summary here: https://forum.inaturalist.org/t/penang-intersecondary-school-city-nature-challenge-project/364/45?u=tiwane
Related to this issue, but somewhat different is the submission of observations for training purposes. This example seemed fishy because, while the identification was correct, the observation date/time was within a minute of the submission date/time. That doesn’t make sense for a pinned specimen on display.
Only after I inquired about that did the submitter explain that the submission was made in a training exercise. So the location isn’t accurate either. I gave it a thumbs down on the accuracy of the date and location, but it’s really easy to assume that cases like this are genuine.
I have seen other cases like this, so I think it would be useful if there was a sandbox version of iNaturalist available to the public where training exercises like this could take place, but the data is isolated from genuine observations (not to mention GBIF).
As someone who probably would have stopped using the app and probably not bothered at all with iNat if I had been prevented from adding more than 10 obs my first day, I think, as @octobertraveler said, that lumping people like us in with students and duress users will do more harm than good. Eventually I expect the community want people like us and keep us engaged so we get better and have more to contribute. Put yourself in the position where you are excited to use the app (I used ebird mostly before this and still do FYI and they didn’t have any limit to how much data you submit) go out in your patch or a rail trail and then have to carefully choose which things to report…based on what? That requires focus be put in the wrong place in my view. The number of obs is only going to deter users trying to populate the site with crappy data for some other purpose: school, a challenge, etc. For me what would have been the rationale? If my 10 obs are garbage because I don’t know what I’m doing but I have to wait to try again until tomorrow and keep track of when I can do more that seems silly to me and would have felt that the emphasis was on keeping me out rather than cultivating my education and better site usage.
The idea of a training zone sounds good to me. I really thought that Discourse’s discobot interactive welcome was great, in that it works with you to learn the site and corrects you if you’ve done something wrong. If iNat had some onboarding feature that interactively walked you through your first few observations that would be useful and might in of itself deter users who are impatiently just trying to compete with classmates since you’d have to go through the tutorials if you want to make IDs and obs. it would prevent some issues with poor photos (although there are no rules against blurry photos and the issue is the bad IDs as far as data is concerned not a photo that nobody IDs and sits there), incorrect use of Identitron, info on geoprivacy etc.
I know I am not a tech person so maybe this isn’t feasible. If all new users did have to look at examples of bad observations vs. ideal ones this might teach better than the current cartoon instructions (although those are cute they don’t show how a photo can help or hurt.) which one needs to have the initiative to seek out rather than it being an automatic, compulsory thing when signing up.
We have to remember to “assume most users mean well” and address the problem at hand which is not amateurs sincerely trying but those forced or just using it like a game not a serious (but fun!) data repository for citizens and pros alike.
A welcome wagon of interested moderators is a lovely idea and the human touch to curb bad behavior might create more of a sense of accountability. I still make mistakes or am careless with buttons and IDs if I’m tired but I am around to correct them and apologize (too much usually). I personally respond(ed) well to those interactions and hopefully my data has been good for the site as much as possible and not bad for it.
I’ll stop there but one more plea not to lump users in a punitive way but rather find new ways to encourage and educate and weed out duress users.
My understanding is that students are posting their observations to projects for the teacher to see for a grade so, at least for identifications, I wonder if it might also be useful to tackle this issue by looking at project usage as well.
I don’t know how easy either of these might be to code, but two possibilities are:
A) The creation of ‘School Projects’ that work differently than standard projects. For example, identifications from project members might not count toward CID on project observations unless approved by the project creator or a project administrator. I imagine this would cut down on the number of RG observations that are the result of students using the AI and/or blindly agreeing with one another.
Going an extra step further, if observations were not officially tallied in the project numbers until they were reviewed by the creator/administrators it might also bring inappropriate attention from students to a teacher’s attention more easily. And if the students notice that their observations haven’t been added to the project tallies yet it might encourage them to notify their teacher and ask that the recent additions be gone through and approved. (Thus the students would care as much about the teacher’s attention to the project as regular iNat users who might also be going through the observations.)
Perhaps when a project is created the site can look for a list of keywords (school, class, elementary, primary, etc.) with a pop-up asking if it’s for a class project and explaining how it works. That might not work in every case but perhaps it could be a start?
B) Another option is to not have project members’ identifications on other project members’ observations count toward CID if their account is below a certain age. They would still be allowed to add full identifications to other observations on the site as normal, the restriction would only be for the projects that they have voluntarily joined. This wouldn’t do much to help in collection projects – which includes the City Nature Challenges, if I remember correctly – but might help for teacher-created class projects.