Annotators, what is your workflow?

General
1

I’m not proud to admit that I usually do not annotate (neither my own observations nor the ones that I identify). I either simply forget (on my own observations) or I’d rather spend my time doing more IDs.

However, I see a lot of you have made an insane number of annotations. Thank you for that, first of all!! It’s really impressive, in my opinion, because compared to most IDs I make, it takes significantly more time. A while ago, I learned about the keyboard shortcuts for annotation, but I still find that I’d need to spend maybe 5x as much time per observation as compared to just leaving an ID.

Now that winter is approaching and both my time in the field as well as the time I spend IDing are (likely) to decrease, I thought it a good opportunity to change my ways and become a better annotator. So I wanted to know: What is your annotation workflow? Do you have any tips/tricks for an “aspiring annotator”? How much time do you spend per observation?
Please share your secrets! :p

2

Have a look at my journal post - The power of annotations

3

Since iNat is meant to be fun – choose a taxon where you already make use of annotations. For my plant focus that started with
Show me what this taxon’s flowers, or fruit, look like ? Someone has to make the annotations.
Also if you use phenology graphs to see when … there are 300 obs, but only a few say Flowers !

You might prefer Adult vs Nymph / Larva or …

Start with a taxon that needs only some annotations, give yourself an achievable target. I choose to annotate what I find useful. No need to fill in ‘all’ the annotations for a given obs.

Another would be adding useful annotations to your own obs. I went thru mine and added ‘flowers’.

4

I always annotate my own observations, and I try to add annotations to any observations that I ID.

A couple of years ago I read on here someone suggesting that adding life stage to Lepidoptera was a useful thing to do, so I started doing that for all Australian observations where the observer hasn’t done it themselves.

I was using keyboard commands, but recently started using the iNat metadata tool browser add-in which allows bulk annotating:

https://forum.inaturalist.org/t/official-release-of-the-inaturalist-metadata-tool-by-megachile/79300

It is brilliant!

5

It is brilliant! @Megachile rocks!!

6

Whenever I get curious about when a plant flowers or what the flowers / seeds look like I’ll annotate that species in the identify page, I started annotating every Eucalyptus observation in TAS and I have done half (~4000) so far. If there’s less than 100-200 observations it’s easy to knock out a whole species in a short period of time while listening to audio

7

Sometimes in the identify module I see something I’m interested in knowing more about. (I use the word “sometimes” very loosely here…) I click through to the information page, and look at the life stage graph (or another detail)

image
image602×396 37.4 KB

If the “No annotation” number is high, then …

image
image439×224 21.3 KB

…I add annotations :)

8

It’s definitely a very helpful thing to do. I’m constantly amazed that folks who really should know better will go out of their way to apply very specific IDs (down to subspecies), but don’t bother to add a life stage annotation. A long time ago, those of us working on an Atlasing project for butterflies in our region opted to treat all observations that have no life stage annotation as adults by default (to save time/effort that would otherwise be required since most observations are for adults). In contrast, we were pretty religious about assigning life stage annotations to all non-adults. Now that we have the metadata tool, it would probably be a lot less work to add “adult” to these observations, but it would still require a non-zero effort. I have found that the metadata tool is very powerful, but with that power comes the possibility of making errors (eg. by hitting the wrong keyboard shortcut). Because I’m usually doing IDs and adding other annotations at the same time, I don’t want things to get too complicated and accidentally apply the wrong annotation to a bunch of observations. So continuing to leave observations of adults without a life stage annotations saves some time/effort, and “derisks” my annotation workflow.

Circling back to the OPs question:

Because I’m both an identifier and a data consumer, the annotation I do is self-serving. That others may benefit from my effort is a bonus.

Personally, I have a flexible definition of “annotation” which includes adding various observation fields that perform similar functions to the official annotations (sensu stricto). This is because I use observation fields to correct/augment annotations that have been set incorrectly (or are inherently incomplete). For example, if the life stage annotation is set incorrectly, I set the “Insect Life Stage” field to the correct value. When my software harvests data from the iNat API, it gives priority to the field. The sex annotation only allows male/female, but for observations that show both sexes, I can add the Sex observation field with a value of “mixed”.

Having buttons/shortcuts defined in the Metadata tool for the various annotations I want to apply speeds things up immeasurably, and allows me to apply annotations that I would not have dreamed of applying in the past (even when I’m only applying them one observation at a time). I mostly do annotations at the same time as I’m doing identification, but for certain assessments that require judgement, I find it advantageous to perform these annotation tasks in isolation. For example, I might filter out a set of observations and add a particular annotation/field and nothing else. This helps ensure I’m performing the assessment in a consistent manner across all observations under consideration, and reduces the chances of applying the wrong shortcut.

I have a second step in my work flow, which is to audit the annotations at the end of the year (prior to harvesting data). So far, I have only been doing this for Life Stage, but I may consider doing it for Sex as well. What I do is filter all observations within my taxon/region scope based on each Life Stage in turn, and I scan the thumbnails for observations that don’t match the selected life stage. I start with those that have no life stage annotation, and I apply the appropriate life stage annotation to any non-adults I spot. Yes, I could then apply “adult” to all the others in bulk, but each bulk operation would add a not insignificant delay, and I would have to be very careful that I’m not doing it incorrectly. As there’s already hundreds of thousands of observations of adults that don’t have a life stage annotation, it seems pointless to change this approach now. I then review those that have the life stage annotation set. If I see any that are wrong, I downvote the existing annotation, and add the Insect Life Stage field with the correct value. That makes sure that the correct life stage ends up in our database, and it might help others if they clue in that somebody is adding the correct life stage in this field.

9

I find that annotating and identifying are separate tasks for me. Mostly because I have to have different filters selected and I can’t be changing them back and forth all the time.

I annotate life stage for Lepidoptera for southern Africa. It took me only a couple months to catch up on about 50,000 obs using Megachile’s tool, and now I keep up to date every couple days.

I personally find it much more satisfying to have per page set to 100 rather than the default 30. Then I know the size of the task ahead without having to overwork my brain :zany_face:

10

So far I’ve been picking a taxon I’m interested in and an annotation that you can evaluate from them in a glance, setting per_page=200 on Identify, and knocking out a page or three while I’m listening to a youtube video or something off to the side.

Life stage for lepidopterans is pretty easy, but can feel sisyphean with how many unannotated observations there are, so I haven’t done too much of it.

My main thing has been annotating sex for local-to-me bees with very stark sexual dimorphism, purely because I wanted to see on the graphs when the males and females are out so I know when to start looking for each of them next year. There’s less observations to sort through when you focus on one species or one genus in one geographical area, so it feels more doable for one person to tackle. And being more familiar with the taxa means I get to feel clever when I spot something mis-identified in the pile, too.

11

When I have already clicked on, and opened, an observation to ID it, after clicking on the “agree” button for the ID, I move my cursor over to the right side of the screen to where the annotation fields are, and scroll down a bit to get them back up near the top of the page. Then I click on “alive”, then I click on “organism”.

I usually just leave it with those two fields checked, because the things I identify, such as White-tailed Deer and American Pika, usually can not have the sex or the age determined with absolute certainty, so I just leave those fields blank.

However, if I am annotating birds with real obvious sexual dimorphism, such as Red-winged Blackbirds or Harlequin Ducks, then I will also click on either “male” or “female”.

And for the stage of life or whatever it’s called, I will click on “adult” or “juvenile” if that is readily apparent. If it isn’t readily apparent, as it often isn’t with birds seen in September or October, or with deer seen in November, December, or January, then I just leave that “stage of life” field empty because there is no way to know for sure.

Most annotations, if done correctly, will have a couple of the fields left blank, because it is often not possible to know those things from the information we are given in the observation. So don’t ever feel that you need to check every box - that isn’t required or expected.

For each observation that I identify and do annotations for, the ID takes about a second and the annotations take about 3 or 4 seconds. So yes, doing the annotations does greatly increase the amount of time it takes to “complete” an observation, but if you already have the ob open on your screen, why not take those few extra seconds to just click a few things?

Just do the annotations that you can know for sure right away. If one of the annotation questions is confusing or not readily apparent, just leave it blank and move on to the next observation. There’s no need to spend time on the difficult uncertain things when there are so many obs that have real obvious stuff that can be readily checked off.

12

Thanks to all the above folks for your tips, which are very helpful. So far, I’ve added 238,000+ annotations. I annotate my own observations and whenever I ID someone else’s observations, I try to annotate those when possible. My workflow is like many others stated above: I use the Identify tool and search for observations with a trait of interest or a specific taxa (e.g., Trifolium) and work for a few hours while listening to music (annotating takes less brain power than IDing, in most cases).

I focus on taxa that are of interest to me, for example:

13

For butterflies in my region, the backlog of unannotated adults is probably something like half a million. Sure, I COULD knock that off fairly quickly (considering that I’ve already checked and annotated all the non-adults), but there are many other tasks on my to-do list that would yield tangible benefits, whereas this would have no payoff at all (for me).

I’m doing something similar for a particular genus that has several difficult to separate species. I’m currently going over all the observations on iNat and (re)identifying them to species/complex/genus as best I can, as well as adding the Sex annotation, and a field describing the degree of wear (as a proxy for how long the butterfly in question has been on the wing). For new observations, I don’t mind doing the sex annotation at the same time that I do ID work, but I find that I prefer to do the “wear” evaluation as a separate activity. So at the end of the year, I’ll do a sweep of all observations for the genus to add the “wear” field. Eventually, I hope to have a large dataset of iNat observations with sex and wear that I can chart to see if the identifications I have applied “make sense”. Hopefully, this will lead to better defined flight season charts for some species and possibly, improvement in how the IDs are actually made.

Unfortunately, there just isn’t time to do such detailed annotations on all species.

14

how do you set it to 100?

15

Most of my annotations are added via the metadata tool. When I’m identifying Lepidoptera, I have buttons for “larva”, “pupa”, “adult”, “mine”, “web”, and “dead” on the screen.

  • larva adds Larva/Alive/Organism
  • pupa adds Pupa
  • adult adds Adult/Alive/Organism
  • mine adds Leafmine
  • web adds Larva/Construction
  • dead adds Dead

Because the metadata tool doesn’t overwrite existing annotations, clicking Dead before adult or alive renders the “alive” portion of those buttons moot. A quick combo of clicks on one or two of those buttons applies all the relevant annotations to virtually any Lepidoptera observation (for weird edge cases I can do them manually). Since making these buttons, I’ve added tens of thousands of annotations while identifying, without really needing to change my workflow at all, besides the quick button click before moving to the next observation. If I had to actually move over to the annotation tab and manually add them, I don’t think I’d be annotating much at all, as I think my time is better spent identifying.

16

how do you set it to 100?

Add &per_page=100 to the URL

17

I also have a journal post regarding annotation of insects: https://www.inaturalist.org/journal/michaelpirrello/133851-guide-for-annotating-insects-on-inaturalist-draft

It may seem overly simplistic, but I find a lot of people annotate inappropriately. For example, adding an annotation where a track or sign implies the presence of particular life stage, but the life stage is not actually present. This can skew the life stage graph and actually negatively impact the value of the data in my opinion. Following the guide in the post should make sure that all the appropriate annotations get applied, and no inappropriate ones.

18

It does overwrite an annotation when it’s your own, no?

19

I’ll be checking out all these suggestions which look awesome, but just wanted to put in a pitch for NOT WORRYING ABOUT whether it takes longer to do annotated IDs than unannotated ones. Are you willing to drive 2 hours to get to a great trail? Are you willing to sit quietly waiting for pollinators to appear? Are you willing to spend half an hour slogging through a dichotomous key? Sure you are, because it’s fun and worthwhile and rewarding. Honestly, efficiency is overrated.

20

I can’t say I entirely agree with this take. I get that the concern is that this will impact phenology graphs, but this seems like a wasted opportunity to allow for more precise filtering.

For many species, empty cocoons, larval molts, empty egg masses, larval cases, etc. are present year-round. If I filter for a life stage, I’d expect to see all the images associated with that life stage, whether or not the insect is there. Caterpillar frass, leaf mines, actual caterpillars, feeding damage, larval molts, etc. are all observations of larvae. If you want to see only observations with the larva itself present, that’s what EoP: Organism exists:

  • To understand when actual larvae are present, use EoP: Organism combined with Life Stage: Larva.
  • To see when evidence of larvae can be observed, use just Life Stage: Larvae without the EoP filter.

Your guide seems to indicate that Life Stage should only exist as an annotation in the subset of observations where EoP: Organism is selected, which would make that second type of search impossible. Molts couldn’t be annotated as larval or pupal molts; constructions couldn’t be annotated as being adult constructions or larval ones; leaf mines made by larvae couldn’t be annotated as distinct from those made by adult insects… It seems like such a massive wasted potential for further precision in data sorting, and the main argument I see against it is “phenology data”, which again is easily solved by just using EoP: Organism as a filter for phenology queries so the old molts, empty cocoons, abandoned mines, etc. aren’t included.

I’ve also read through some old threads on the question, and the other more technical argument that seems to come up is “you can only annotate based on what you see in the observation”. This is true of IDs too though. But if I can see an empty spongy moth pupa, identifying it as a spongy moth (despite there being only evidence of the organism, not an actual organism) seems to me exactly the same as annotating it as pupa (despite there being only evidence of a pupa, not an actual pupa). So I don’t find that argument very convincing either.

I guess the strongest argument I can come up with against annotating life stage of diverse EoPs is that the default phenology graphs fail to filter by anything other than the Life Stage annotation, so they might be a little misleading if indirect evidence of a life stage is showing up after that life stage is done for the year. But tbh, the fact that dead organisms still show up on those graphs means they’re already going to be a bit misleading. After all, I could go photograph a dead adult moth in my windowsill right now that only flies in April, and it would still be correct to annotate it as an adult in November. If anything, it speaks more to those graphs being overly simplistic in their filtering than anything else.

Personally, if it’s wrong to annotate a caterpillar nest full of caterpillar molts and caterpillar frass as “larva”, but it becomes acceptable to do so if you dig through it and find one full months-old dried caterpillar corpse somewhere in there, then I don’t want to be right. If it’s the position of staff that no life stage should ever be selected unless the EoP is “Organism”, then I’ll make a feature request to lock life stage unless EoP: Organism has been selected, because that interpretation seems very counterintuitive to me (and apparently to many others, based on how many observations have a life stage selected when the EoP is molt, leafmine, frass, etc.)