iNaturalist failing beyond reason for Nepytia phantasmaria

Bug Reports
1

Please fill out the following sections to the best of your ability, it will help us investigate bugs if we have this information at the outset. Screenshots are especially helpful, so please provide those if you can.

Platform (Android, iOS, Website): Website

App version number, if a mobile app issue (shown under Settings or About):

Browser, if a website issue (Firefox, Chrome, etc) :

URLs (aka web addresses) of any relevant observations or pages:

Screenshots of what you are seeing (instructions for taking a screenshot on computers and mobile devices: https://www.take-a-screenshot.org/):

Description of problem (please provide a set of steps we can use to replicate the issue, and make as many as you need.):

Step 1: Check any observation of a Nepytia (Geometridae moth) west of the Cascade Mountains of the Pacific Northwest of North America.

Step 2: Check iNaturalist’s suggested ID’s - Every single one will suggest Nepytia canosaria as the top suggestion, a species that lacks the black, double forked vein lines in the middle of the forewing, of all of these Nepytia observations, and N. canosaria a species of not occurring west of the Rockies, and will never say that N. phantasmaria is both “visually similar”, and “seen nearby”, even though it has those double forked, black vein lines in the middle of the forewing, that only N. phantasmaria has, with N. phantasmaria the only Nepytia species west of the Cascades. This is currently one of the most common moths west of the Cascades, and correcting iNaturalist errors could be a full time job, taking me away from many other ID’s

Step 3:

2

You can add it here. https://forum.inaturalist.org/t/computer-vision-clean-up-wiki/7281
I don’t see a bug, the genus consists of quite similar for AI species all but 1 with not many photos, I tried some obs and AI suggests genus and some species, sometimes even abother genus, phantasmaria wasn’t the first one suggested, but it was in the list.

5 Likes
3

I agree, I think this is a bias in the model becaus there are so many more photos of N. canosaria. We will start training a new model soon, so fixing the IDs soon would be helpful. If you have comparison photos of the diagnostic area I’d be happy to try and fix some IDs.

5 Likes
4

@tiwane the diagnostic area is easy to see once you know where to look. Both Nepytia canosaria, and N. phantasmaria have thick, black jagged lines, perpendicular to the line of the forewing, one about 1/3 from the base of the wing, and one about 1/3 from the end of the wing, but only N. phantasmaria has a strong, black doubly forked vein line connecting the two, only hard to see in the most worn individuals. The photos on the species pages I linked to provide easy examples to see that doubly forked vein line present in phantasmaria, and absent in canosaria. It is interesting that there aren’t more phantsmaria observations, as this is currently possibly the most commonly observed moth in my area. I would suspect it is a year that they are peaking.

2 Likes
5

I now see that large part of the reason for the relatively large number of canosaria observations is the much larger range of canosaria through the most populated part of North America, and the narrow range of phantasmaria, which I have dubbed an “endemic”, for its narrow range, restricted to the area west of the Cascade Mountains. It could still be a peak year.

6

The last model update had this requirement:

a taxon must have at least 100 verifiable observations and at least 50 with a community ID to be included in training

and it only

used images from observations meeting the criteria above on September 29, 2019.

There were 99 verifiable observations on 9/28 and 103 on 9/29, so it’s quite possible it simply missed the cut by a few hours. (Or it’s possible the >50 community IDs were reached on those observations after that time. I didn’t look deeply enough into this to tell.)

1 Like
7

It is evidently an outbreak year for this species. I checked August 26 to September 7 for 2020, 2019, 2018, 2017, 2016, and 2015. This year we have had 64 observations in that period. The highest in other years was 8 observations, and other years something like 0 - 2.

1 Like
8

The userbase is constantly growing so you should account for that when comparing observation number growth.

Also, AI failing to act “reasonably” is not a bug, unless you think that AI has become as good as human? People are beginning to expect too much from AI. The computer vision needs to be trained with correctly identified observations (by human experts) in order to correctly identify observations. As @aisti has noted, probably the species has not even made it to the computer vision model so it cannot possibly be among the suggestions.
Which begs the question - is there a way to see which species have made it to the computer vision model?

3 Likes
9

If @fffffffff has seen it in the suggestions list, then it must have made it to the model I guess. But probably until it got there there were already lots of wrong IDs. Still, it would be nice to have a way to check which taxa are in the model?

1 Like
10

@melodie_96 I have now added this species, and my description of the issue, to the computer vision clean up place you linked to. I hope it isn’t a duplication. I will add that phantasmaria hasn’t always made it on the list of iNaturalist suggestions, and part of my problem is that in checking many iNaturalist suggestions iNaturalist has never said that phantasmaria is both “visually similar”, and “seen nearby”.

11

Could the AI perhaps be made to exclude species whose distribution doesn’t match (i.e. suggestions are only provided after the locality is filled in)? For example very freqently North American grasshopper nymphs are misidentified as Pezotettix or Pezotettix giornae, a common brachypterous species of this genus which is restricted to the Mediterranean.

12

There’re numerous topics discussing that and for a fast answer now it couldn’t.

2 Likes
13

@tiwane In the mid-Atlantic region, we have an undescribed Nepytia species that is common and diagnosable from photos. Is there a way to have the next CV training session attempt to recognize this species as distinct from the others, to help reduce the Nepytia confusion in the current model?

14

I think this is possible only if a curator adds a taxon representing the undescribed species (unlikely in my opinion), and you reach the requirements for computer vision listed above: 100 verifiable observations of your species, with at least 50 having a community ID.

1 Like