There Was An Attempt: Hardest and Easiest Butterflies/Moths to ID in the U.S.A

[comradejon]

Fellow iNatters!

Are you itching to start identifying butterflies and moths: (1) because you’re curious and it’s cool, or (2) to fix your embarrassing submitted-observations-to-identifications ratio?

Are you scared of being publicly humiliated by being wrong? Or are you addicted to identifying and looking for your next challenge?

Do I have some data for you! For dozens of minutes, I endured grueling data entry whilst telling my computer’s calculator what numbers to add. And then I wrote this post. All for you!

Methodology: I took the top 50 U.S.A. butterfly and moth species (by total research-grade observations), tallied the total U.S.A. mistaken ID’s noted in the “similar species” section for each species - such a helpful learning feature! - and then figured the error rate for each species.

Here are the highlights:

THE EASIEST TO IDENTIFY
(1) Ailanthus Webworm Moth - Error Rate: 0.20%

Take it from me - any dummy can ID this species! And you should, too!

(2) Silver-spotted Skipper - Error Rate: 0.53%

Now that I’ve flagged two of its subspecies…oh wait, you’re not interested! Hey, rumor has it that those who ID this species wind up with a little silver themselves!

(3) Mourning Cloak - Error Rate: 0.56%

This is not a sad species, but instead represents the opportunity for an identifying renaissance; a blossoming of identifying prowess!

(4) Red Admiral - Error Rate: 0.62%

If you build your ID’ing muscles, the harder stuff gets easier!

(5) American Snout - Error Rate: 0.65%

I can unequivocally say we can all use a little more snout in our lives! I don’t know what that means, but I can say it. I more seriously say: consider ID’ing this relatively easy-to-ID species!

THE HARDEST TO IDENTIFY
(1) Virginia Tiger Moth - Error Rate: 18.37%

Oh god no. That is not a typo - 18.37%.

(2) Spicebush Swallowtail - Error Rate: 10.95%

Hours of arduous study may result in you having a lower error rate than the general public!

(3) Salt Marsh Moth - Error Rate: 9.33%

If you’d like to try extra-hard to not be wrong, you might consider ID’ing this species!

(4) Fall Webworm Moth - Error Rate: 8.46%

It’s always the right time of year to try learning how to ID the Fall Webworm Moth!

(5) Carolina Sphinx - Error Rate: 7.16%

You, too, can go to Carolina in your mind, by ID’ing this species!

So, was my methodology perfect for determining the hardest and easiest species to ID? Of course not, for so, so many reasons!

For starters: (1) some species are easy to ID in adult stage, but not in larva stage (or vice versa), and this data doesn’t tease that out, and (2) the “similar species” section of each species page is capped at 24 species, which means this data under-represents the number of errors for some species. And it’s worth noting there are lots and lots of easy and difficult species worthy of ID outside the top 50 most observed!

But look, one of the reasons iNaturalist is cool is that any rando can look at the data and find some neat stuff. Also, we often talk on the forum about how to grow the ID’ing community, and encounter people wondering where to start. There are lots of good ways to think about where to start, but easy/hard is one decent way of doing it. So - I thought this was worth sharing!

Surely some of you (only the really awesome folks) want to see the full data set. Here you go! (Whew, my longest post ever)!

Error Rate for all 50 Species

Species	Observations	Total Errors	Error Rate
Ailanthus Webworm Moth	41443	81	0.20%
Silver-spotted Skipper	57885	306	0.53%
Mourning Cloak	31765	179	0.56%
Red Admiral	84289	521	0.62%
American Snout	24105	156	0.65%
Gray Hairstreak	57967	518	0.89%
Gulf Fritillary	100194	1005	1.00%
Monarch	249067	2928	1.18%
White-lined Sphinx	51859	672	1.30%
Green Cloverworm Moth	30890	404	1.31%
Common Buckeye	73188	999	1.36%
Celery Leaftier Moth	20192	279	1.38%
Eastern Tailed-Blue	37963	604	1.59%
Variegated Fritillary	42915	688	1.60%
Milkweed Tussock Moth	21581	380	1.76%
Sleepy Orange	21593	417	1.93%
Cabbage White	65691	1306	1.99%
Spongy Moth	23759	476	2.00%
Red-spotted Admiral	59865	1228	2.05%
Isabella Tiger Moth	50515	1037	2.05%
Hickory Tussock Moth	21922	482	2.20%
Luna Moth	40067	921	2.30%
Hackberry Emperor	35779	937	2.62%
Queen	38992	1025	2.63%
Polyphemus Moth	42899	1296	3.02%
Eastern Tent Caterpillar Moth	30838	934	3.03%
Checkered White	23245	728	3.13%
Great Spangled Fritillary	29931	1018	3.40%
Cloudless Sulpher	26957	997	3.70%
Question Mark	35687	1345	3.77%
Pipevine Swallowtail	57616	2194	3.81%
Fiery Skipper	69694	2694	3.87%
Zabulon Skipper	32552	1276	3.92%
American Lady	45224	1791	3.96%
Painted Lady	62718	2498	3.98%
Pearl Crescent	60090	2435	4.05%
Giant Leopard Moth	25997	1074	4.13%
Peck’s Skipper	21643	911	4.21%
Snowberry Clearwing	20250	926	4.57%
Viceroy	26153	1201	4.59%
Banded Tussock Moth	28975	1344	4.64%
Black Swallowtail	80219	3824	4.77%
Eastern Tiger Swallowtail	103718	5457	5.26%
Huron Sachem	49853	2860	5.74%
Orange Sulphur	35965	2124	5.91%
Carolina Sphinx	20935	1499	7.16%
Fall Webworm Moth	31321	2650	8.46%
Salt Marsh Moth	37997	3544	9.33%
Spicebush Swallowtail	39770	4354	10.95%
Virginian Tiger Moth	22652	4162	18.37%

[DianaStuder]

Nicely presented!
Here hoping for lots of eager identifiers

[spiphany]

@comradejon

(2) the “similar species” section of each species page is capped at 24 species, which means this data under-represents the number of errors for some species.

I believe it also only includes active wrong IDs – i.e., it does not include cases where a user has withdrawn their ID after getting feedback from other users. This means that the rate of misidentification may be higher depending on how diligent users are about responding to corrections.

[comradejon]

Aha, that’s another good thing to consider - I wasn’t sure how it worked exactly!

[theo_fiedler]

Haha, this is great! Thanks for the tier list, good read ;)

[sbasalaev]

@spiphany

I believe it also only includes active wrong IDs

The neat thing is that you can click on that number of errors and see all those confusing observations for yourself. If I do that it shows observations with both active disagreements and withdrawn identifications so number of errors includes both.

[ellyne]

This is really cool!
I was wondering when you were going through the research grade observations, if you saw the caterpillar/larva stage contributing to the ID error rates for some of the species? Or did you filter out the larva stage?

Sometimes when I’m identifying a new moth and check out the “Similar Species”, I see vastly different looking adult moths that I don’t think the CV or a person would be confused with, but with closer inspection, I notice it was because the caterpillars were visually similar, not the adult stage. Either way, I love this and thank you for the hard work!

[paul_dennehy]

Very neat!
I think I can add some context for why some of these are so commonly confused:

@comradejon

Carolina Sphinx 20935 1499 7.16%

These are the common “hornworm” on tomato plants over most of the eastern USA, but a different species in the genus, M. quinquemaculata, is usually given the common name “tomato hornworm”. M. quinquemaculata is uncommon in most of the eastern USA, but observers see a hornworm on their tomato and say “look, a tomato hornworm”, which technically speaking is usually wrong.

@comradejon

Eastern Tiger Swallowtail 103718 5457 5.26%

@comradejon

Black Swallowtail 80219 3824 4.77%

@comradejon

Spicebush Swallowtail 39770 4354 10.95%

@comradejon

Pipevine Swallowtail 57616 2194 3.81%

@comradejon

Red-spotted Admiral 59865 1228 2.05%

These are part of a mimicry complex, so they really want to be confused, so to speak.

@comradejon

Snowberry Clearwing 20250 926 4.57%

One of two common Hemaris in the eastern USA, both equally common. Most people have heard of the other one, the “Hummingbird Clearwing”, because the common name is so descriptive (they look like hummingbirds!). But if you call all your Hemaris “Hummingbird Clearwings”, you’ll be wrong about 50% of the time.

@comradejon

Virginian Tiger Moth 22652 4162 18.37%

@comradejon

Fall Webworm Moth 31321 2650 8.46%

Generally you need to see the abdomen to identify these, and most live photos don’t show the abdomen, so they’re inherently unidentifiable. There are some active identifiers who kick them back to tribe Spilosomina routinely.

@comradejon

Salt Marsh Moth 37997 3544 9.33%

@comradejon

Virginian Tiger Moth 22652 4162 18.37%

These have very similar larvae that the CV mixes up quite often, even though the adults are very different.
The fact that S. virginica has confusion of both adults and larvae makes it unsurprising that it’s the most mis-identified.

@comradejon

Banded Tussock Moth 28975 1344 4.64%

Inseparable from the Sycamore Tussock moth as an adult unless you dissect it; probably the most charismatic and common “needs dissection” moth pair in the eastern USA. Also there is a different subfamily, called Lymantriinae, with the common name “Tussock Moths”. The Banded/Sycamore Tussock moths are not in this subfamily, so common-name-users often put these in the wrong subfamily, thinking they’re using a broad taxon that includes this genus.

@comradejon

Orange Sulphur 35965 2124 5.91%

White form females are often inseparable from Clouded Sulfurs, and even the yellow forms are difficult to separate from underside-shots, which are typically what get posted.

[danly]

I’m convinced that no one has more fun crunching numbers than you do, and the way you present your results is always delightful.

And, as an added bonus, those of us of a certain age get the “ear-worm” of the day: “Carolina in My Mind.”

Thanks!

[spiphany]

@sbasalaev

If I do that it shows observations with both active disagreements and withdrawn identifications so number of errors includes both.

That’s odd, because the results I see only appear to include active IDs, even for species that I know are frequently mis-ID’d. If withdrawn IDs were included, the numbers would be higher.

[comradejon]

@ellyne

I was wondering when you were going through the research grade observations, if you saw the caterpillar/larva stage contributing to the ID error rates for some of the species? Or did you filter out the larva stage?

Regarding your first question: Definitely! The caterpillar/larva stage is a very big factor contributing to error rates for some species! One pair that especially stood out to me:

@comradejon

Luna Moth 40067 921 2.30%

@comradejon

Polyphemus Moth 42899 1296 3.02%

As adults, these are unmistakably different-looking (and easy to ID):

But they’re each other’s most-mistaken ID (by a huge margin), because:

Regarding your second question, I didn’t filter out the larva stage - for a few reasons!

• It didn’t occur to me until I was a half-dozen species deep, and I couldn’t possibly bear starting over.
• In addition to starting over, it would have involved a lot more button pushing, which my delicate sensibilities couldn’t handle.
• Our butterfly and moth species are woefully under-annotated, and annotated very unevenly across species, and regardless of annotation, some species have commonly-found larva and others not. So it just seemed…complicated.

[comradejon]

@paul_dennehy

I think I can add some context for why some of these are so commonly confused:

That was incredibly helpful - thanks!

@sbasalaev

The neat thing is that you can click on that number of errors and see all those confusing observations for yourself.

I never realized that - surely I’m not alone. Thanks for mentioning it!

@spiphany

That’s odd, because the results I see only appear to include active IDs, even for species that I know are frequently mis-ID’d. If withdrawn IDs were included, the numbers would be higher.

I agree - it looks to me like it’s just querying active mistaken IDs, although some of the observations have both active mistaken IDs and withdrawn mistaken IDs.

[Vireya]

This was a really fun post, even though I am not likely to ever be identifying any USA butterflies or moths.

[michaelpirrello]

Some of you may remember this post from a few years back: https://forum.inaturalist.org/t/what-things-are-misidentified-as-large-milkweed-bug/12571

If you’re interested, you can do the Sankey diagrams on those hardest to ID species to figure out what they’re being mis-identified as.

[jasonhernandez74]

@danly

the “ear-worm” of the day

Speaking of ear-worms, I wonder what the error rate is for Helicoverpa zea?

[alanliang]

This is a very interesting list, but it is probably misleading to say that high error rate = hard to identify. The truly difficult to ID species would not even be identified to species level, and would therefore not make the list. Many of the species on this list are very much identifiable to species – in fact, that’s probably why they’re on there at all.

The species with the highest error rates aren’t necessarily difficult to ID, but are ones that tend to have a high disparity between the accuracy of expert IDs and non-expert IDs. In other words, species that are actually quite easy to identify to species by experts but are frequently misidentified by non-experts for reasons such as the ones @paul_dennehy has mentioned (bad common names, mimicry complex). It takes three correct IDs to overturn one wrong ID, after all. Truly difficult species are rarely able to reach such a consensus.

[comradejon]

@jasonhernandez74

I wonder what the error rate is for Helicoverpa zea?

That would be a very reasonable 1.46%!

@alanliang

it is probably misleading to say that high error rate = hard to identify

I’d say: yes and no. It’s true the most difficult species to identify wouldn’t even make the top-50 research-grade list. And since any member of the public can (and often does) make IDs, the data definitely can’t really speak to how easy experts find something to identify. And of course, there are so many other issues, big and small. Some more examples:

• Species with many prompt identifiers are probably more likely to have a misidentification withdrawn (it’s mentally easier to ignore one person correcting you two months later than to ignore six people immediately telling you you’re wrong), which would mean it wouldn’t show up on the “similar species” list.
• Some species share similar names with completely unrelated species; is it a meaningful misidentification if somebody just typed in the wrong name? It’s hard to say that speaks to the “difficulty” of identifying.

@alanliang

The species with the highest error rates aren’t necessarily difficult to ID, but are ones that tend to have a high disparity between the accuracy of expert IDs and non-expert IDs.

See, this is why I think there’s a general kernel of truth in the data! I’d say the species that have a high disparity between the accuracy of “expert” IDs and “non-expert” IDs are exactly the species that are the most difficult to ID - because those take the most training to learn!

But then we’re deep into a philosophical discussion of what makes something “difficult,” which no amount of iNaturalist data will be able to answer.

So, there are a ton of flaws and caveats with this sort of analysis - absolutely - but as a measure of the relative difficulty of identifying the 50 most-commonly identified U.S.A. moths and butterflies - eh, it’s a pretty good starting point. We sure could do worse (I feel like I just dared myself, haha)!

[mo0nsgreenthumb]

This was fantastic! Thank you for all the hard work! Winter is that time of the year where all the pollinators are gone and it’s depressing so I’m focusing this winter on ID stuff, this will help.

[paul_dennehy]

Given that H. zea is externally identical to H. armigera, if range is ignored, the error rate could be as high as 100% for both of them based on live photos. ;) lol

[paul_dennehy]

I agree, this is a good measure of the “hardest-to-identify” of the

@comradejon

50 most-commonly identified U.S.A. moths and butterflies

But yes, of course the list of “completely unidentifiable without dissection or DNA work” moths is nearly endless. Just here in the Northern USA, that includes most of the nondescript Gelechiidae, virtually all Coleophora, most Blastobasids, most Eupithecia, Xanthotype, Carpisona, the shiny metallic Pyla species, Halysidota, many (most?) Bucculatrix and Elachista, Hahncappsia marculenta/neomarculenta/neobliteralis, Eufidonia, most Acleris, Symmerista, Anania tertialis/plectilis, just to name a few… But these should mostly be left at “species complex” or genus level on iNat, as most identifiers are aware that they can’t be separated from photos of live adult moths. In some cases they’re among the most abundant moths at the lights, but the individual species will always be underrepresented on iNat because most observers aren’t collecting the moths and dissecting them to get names on them.