Something wrong with GBIF map overlays for numerous insect species

Looking at the distribution map for taxon Eleodes goryi: applying the GBIF overlay throws up a gazillion data points that are not at all in agreement with the data at GBIF.
Safari 14.1.2 interface on Catalina 10.15.7
taxon page: https://www.inaturalist.org/taxa/334271-Eleodes-goryi



The same problem seems to apply to other species in the genus Eleodes, such as E. osculans (but I confess that I do not have time to test them all). Possibly, the genus level data is what is being displayed incorrectly on the species level map overlays.
The problem does NOT apply to an arbitrary plant taxon, Sanvitalia aberti. And the problem does NOT apply to an arbitrary insect taxon, Egira variabilis.

2 Likes

Eleodes-goryi has been corrected, genus level data was being displayed for the species.


It’s the same issue for E. osculans (not yet corrected), but doesn’t seem to be happening to all species in the genus.

3 Likes

There are a few more species in the genus with the same issue. This is not a complete list, so far I can’t see any pattern linking them:
Eleodes clavicornis
Eleodes inculta
Eleodes littoralis
Eleodes parvicollis
Eleodes pimelioides
Eleodes rotundipennis
Can they be found and fixed automatically, or do they need to be manually checked and corrected by curators?

1 Like

The incomparable Bouteloua showed me how to fix this problem. So this morning I’m going through all the species of Eleodes and checking/repairing the GBIF link. So far about one in 6 or 8 species is compromised in this way; certain subgenera are worse than that. But I do not yet see a pattern to it!
Here the fix is described: https://www.inaturalist.org/flags/544104#activity_comment_0f93c38c-8299-4474-81d0-e9af519c8d62
At least, in the case of a populous genus like Eleodes, it’s rather obvious when the map data is incorrect. But there maybe other taxa for which this is not the case! So it would be useful if anybody can puzzle out the pattern or cause of the issue!

2 Likes

I just repaired GBIF links for 46 species of Eleodes. Identifying taxa with incorrect links to GBIF data remains a problem, though

1 Like

Uh oh. Same problem in genus Schinia

Looks that way; I suppose the thing to do is flag the genus .
Curators can fix the issue species-by-species, as explained here: https://www.inaturalist.org/flags/544104

It is caused by taxa updates done on either site. There is no easy way to find issues as the data on the inat side is stored in a very obscure place to which there is no api connection available.

1 Like

i think it would be technically possible to somewhat automate the search for mismatches by scraping / parsing the HTML in the taxon’s taxon scheme page (ex. https://www.inaturalist.org/taxa/326191/schemes) and then seeing if the GBIF taxon has the same name (ex. https://api.gbif.org/v1/species/9734350). that said, you would have to do this one taxon at a time, and it would not be a super efficient process.

if someone had access to the iNat database, or if iNat staff included GBIF ID in their taxon exports or taxon API, then you could get GBIF IDs for all taxa at the same time (or at least in chunks), but then you would still have to look things up over at GBIF, and you’d still have to do that in chunks, as far as i’m aware. so still not a super efficient process.

now for some speculation… i’m not familiar with the process for creating new species, but i wonder if it involves using the parent (genus) as the template for the new species? if so, that might explain how the genus-level GBIF ID might end up on multiple species, and in such a case, i guess you would just want to either procedurally or technologically prevent the GBIF ID from getting copied from another taxon.

not sure if there’s really still a bug here though. seems like a data problem (which for the original reported problem has already been fixed), and even if the source of the bad data could be prevented with a code change, that might be more of a feature request than a bug fix.

1 Like

Here are all the wrong or missing Schinia:

sciname scheme_URL GBIF_id
Schinia saturata https://www.inaturalist.org/taxa/230560/schemes 2773644
Schinia pulchripennis https://www.inaturalist.org/taxa/82483/schemes 4405135
Schinia alensa https://www.inaturalist.org/taxa/82486/schemes 4405135
Schinia aden https://www.inaturalist.org/taxa/82487/schemes 4405135
Schinia ar https://www.inaturalist.org/taxa/82488/schemes 4405135
Schinia antonio https://www.inaturalist.org/taxa/82489/schemes 4405135
Schinia albafascia https://www.inaturalist.org/taxa/82490/schemes 4405135
Schinia acutilinea https://www.inaturalist.org/taxa/82491/schemes 4405135
Schinia approximata https://www.inaturalist.org/taxa/82492/schemes 4405135
Schinia accessa https://www.inaturalist.org/taxa/82493/schemes 4405135
Schinia amaryllis https://www.inaturalist.org/taxa/82494/schemes 4405135
Schinia alencis https://www.inaturalist.org/taxa/82495/schemes 4405135
Schinia intermontana https://www.inaturalist.org/taxa/129559/schemes 4405135
Schinia suetus https://www.inaturalist.org/taxa/129560/schemes 4405135
Schinia arcigera https://www.inaturalist.org/taxa/143196/schemes 4405135
Schinia florida https://www.inaturalist.org/taxa/143316/schemes 4405135
Schinia rivulosa https://www.inaturalist.org/taxa/143382/schemes 4405135
Schinia avemensis https://www.inaturalist.org/taxa/230520/schemes 4405135
Schinia bicuspida https://www.inaturalist.org/taxa/230521/schemes 4405135
Schinia bifascia https://www.inaturalist.org/taxa/230522/schemes 4405135
Schinia bimatris https://www.inaturalist.org/taxa/230523/schemes 4405135
Schinia bina https://www.inaturalist.org/taxa/230524/schemes 4405135
Schinia carminatra https://www.inaturalist.org/taxa/230525/schemes 4405135
Schinia carolinensis https://www.inaturalist.org/taxa/230526/schemes 4405135
Schinia chrysellus https://www.inaturalist.org/taxa/230527/schemes 4405135
Schinia coercita https://www.inaturalist.org/taxa/230528/schemes 4405135
Schinia crenilinea https://www.inaturalist.org/taxa/230529/schemes 4405135
Schinia cumatilis https://www.inaturalist.org/taxa/230530/schemes 4405135
Schinia diffusa https://www.inaturalist.org/taxa/230531/schemes 4405135
Schinia gaurae https://www.inaturalist.org/taxa/230533/schemes 4405135
Schinia gracilenta https://www.inaturalist.org/taxa/230534/schemes 4405135
Schinia grandimedia https://www.inaturalist.org/taxa/230535/schemes 4405135
Schinia honesta https://www.inaturalist.org/taxa/230536/schemes 4405135
Schinia hulstia https://www.inaturalist.org/taxa/230537/schemes 4405135
Schinia illustra https://www.inaturalist.org/taxa/230538/schemes 4405135
Schinia indiana https://www.inaturalist.org/taxa/230539/schemes 4405135
Schinia jaguarina https://www.inaturalist.org/taxa/230540/schemes 4405135
Schinia lucens https://www.inaturalist.org/taxa/230541/schemes 4405135
Schinia lynx https://www.inaturalist.org/taxa/230542/schemes 4405135
Schinia masoni https://www.inaturalist.org/taxa/230543/schemes 4405135
Schinia meadi https://www.inaturalist.org/taxa/230544/schemes 4405135
Schinia mitis https://www.inaturalist.org/taxa/230545/schemes 4405135
Schinia mortua https://www.inaturalist.org/taxa/230546/schemes 4405135
Schinia nubila https://www.inaturalist.org/taxa/230547/schemes 4405135
Schinia nuchalis https://www.inaturalist.org/taxa/230548/schemes 4405135
Schinia nundina https://www.inaturalist.org/taxa/230549/schemes 4405135
Schinia obliqua https://www.inaturalist.org/taxa/230550/schemes 4405135
Schinia obscurata https://www.inaturalist.org/taxa/230551/schemes 4405135
Schinia oleagina https://www.inaturalist.org/taxa/230552/schemes 4405135
Schinia parmeliana https://www.inaturalist.org/taxa/230553/schemes 4405135
Schinia persimilis https://www.inaturalist.org/taxa/230554/schemes 4405135
Schinia petulans https://www.inaturalist.org/taxa/230555/schemes 4405135
Schinia reniformis https://www.inaturalist.org/taxa/230556/schemes 4405135
Schinia roseitincta https://www.inaturalist.org/taxa/230557/schemes 4405135
Schinia rufipenna https://www.inaturalist.org/taxa/230558/schemes 4405135
Schinia sanguinea https://www.inaturalist.org/taxa/230559/schemes 4405135
Schinia scissoides https://www.inaturalist.org/taxa/230561/schemes 4405135
Schinia septentrionalis https://www.inaturalist.org/taxa/230562/schemes 4405135
Schinia simplex https://www.inaturalist.org/taxa/230563/schemes 4405135
Schinia siren https://www.inaturalist.org/taxa/230564/schemes 4405135
Schinia snowi https://www.inaturalist.org/taxa/230565/schemes 4405135
Schinia sordidus https://www.inaturalist.org/taxa/230566/schemes 4405135
Schinia spinosae https://www.inaturalist.org/taxa/230567/schemes 4405135
Schinia tertia https://www.inaturalist.org/taxa/230568/schemes 4405135
Schinia thoreaui https://www.inaturalist.org/taxa/230569/schemes 4405135
Schinia trifascia https://www.inaturalist.org/taxa/230570/schemes 4405135
Schinia tuberculum https://www.inaturalist.org/taxa/230571/schemes 4405135
Schinia ultima https://www.inaturalist.org/taxa/230572/schemes 4405135
Schinia verna https://www.inaturalist.org/taxa/230573/schemes 4405135
Schinia villosa https://www.inaturalist.org/taxa/230574/schemes 4405135
Schinia volupia https://www.inaturalist.org/taxa/230575/schemes 4405135
Schinia erosa https://www.inaturalist.org/taxa/321000/schemes 4405135
Schinia vacciniae https://www.inaturalist.org/taxa/321001/schemes 4405135
Schinia argentifascia https://www.inaturalist.org/taxa/321245/schemes 4405135
Schinia aurantiaca https://www.inaturalist.org/taxa/321246/schemes 4405135
Schinia biundulata https://www.inaturalist.org/taxa/321247/schemes 4405135
Schinia buta https://www.inaturalist.org/taxa/321248/schemes 4405135
Schinia ciliata https://www.inaturalist.org/taxa/321249/schemes 4405135
Schinia cupes https://www.inaturalist.org/taxa/321251/schemes 4405135
Schinia dobla https://www.inaturalist.org/taxa/321253/schemes 4405135
Schinia felicitata https://www.inaturalist.org/taxa/321254/schemes 4405135
Schinia graefiana https://www.inaturalist.org/taxa/321255/schemes 4405135
Schinia intrabilis https://www.inaturalist.org/taxa/321256/schemes 4405135
Schinia jaegeri https://www.inaturalist.org/taxa/321257/schemes 4405135
Schinia ligeae https://www.inaturalist.org/taxa/321258/schemes 4405135
Schinia luxa https://www.inaturalist.org/taxa/321259/schemes 4405135
Schinia macneilli https://www.inaturalist.org/taxa/321260/schemes 4405135
Schinia megarena https://www.inaturalist.org/taxa/321261/schemes 4405135
Schinia miniana https://www.inaturalist.org/taxa/321262/schemes 4405135
Schinia niveicosta https://www.inaturalist.org/taxa/321263/schemes 4405135
Schinia oculata https://www.inaturalist.org/taxa/321264/schemes 4405135
Schinia perminuta https://www.inaturalist.org/taxa/321265/schemes 4405135
Schinia scarletina https://www.inaturalist.org/taxa/321266/schemes 4405135
Schinia sexplagiata https://www.inaturalist.org/taxa/321267/schemes 4405135
Schinia tobia https://www.inaturalist.org/taxa/321268/schemes 4405135
Schinia velaris https://www.inaturalist.org/taxa/321269/schemes 4405135
Schinia varix https://www.inaturalist.org/taxa/334828/schemes 4405135
Schinia unimacula https://www.inaturalist.org/taxa/362566/schemes 4405135
Schinia citrinellus https://www.inaturalist.org/taxa/424508/schemes 4405135
Schinia mexicana https://www.inaturalist.org/taxa/424515/schemes 4405135
Schinia sexata https://www.inaturalist.org/taxa/452641/schemes 4405135
Schinia errans https://www.inaturalist.org/taxa/472780/schemes 4405135
Schinia deserticola https://www.inaturalist.org/taxa/477962/schemes 4405135
Schinia regia https://www.inaturalist.org/taxa/479194/schemes 4405135
Schinia regina https://www.inaturalist.org/taxa/492536/schemes 4405135
Schinia olivacea https://www.inaturalist.org/taxa/506692/schemes 4405135
Schinia hanga https://www.inaturalist.org/taxa/506731/schemes 4405135
Schinia fulleri https://www.inaturalist.org/taxa/507646/schemes 4405135
Schinia edwardsii https://www.inaturalist.org/taxa/230532/schemes 4643743
Schinia subspinosae https://www.inaturalist.org/taxa/1036755/schemes
Schinia immaculata https://www.inaturalist.org/taxa/1103392/schemes
Schinia carmosina https://www.inaturalist.org/taxa/1137202/schemes
Schinia imperialis https://www.inaturalist.org/taxa/1142450/schemes
Schinia maculata https://www.inaturalist.org/taxa/1144136/schemes
Schinia chilensis https://www.inaturalist.org/taxa/1145851/schemes
Schinia bieneri https://www.inaturalist.org/taxa/1245755/schemes
Schinia purpurascens https://www.inaturalist.org/taxa/1245756/schemes
Schinia zuni https://www.inaturalist.org/taxa/1249504/schemes
Schinia rufocostulata https://www.inaturalist.org/taxa/1263313/schemes
Schinia multiplex https://www.inaturalist.org/taxa/1272971/schemes
Schinia pseudomia https://www.inaturalist.org/taxa/1272972/schemes
Schinia subrosea https://www.inaturalist.org/taxa/1272978/schemes
Schinia chanzyi https://www.inaturalist.org/taxa/1272981/schemes
Schinia copiosa https://www.inaturalist.org/taxa/1272984/schemes

Moved this to #curators.

1 Like

Thanks for compiling this list. I could sit down and repair these as soon as I find a chunk of time and the mental space.

3 Likes

Has anybody seen this issue in taxa other than insects?

It’s also a wiki, so anyone can delete species as they are fixed.

i think this kind of thing has been noted twice before in the forum. once was for a mammal, and the other time was for an insect:

I tried to get all the Schinia you listed today. That ate about an hour and a half, but your including the taxon links really sped up the process. Thank you!
The following do not currently (9/10/2021)appear on GBIF: purpurascens, bieneri, chilensis, imperialis, copiosa, chanzyi, subrosea, pseudomia, multiplex.
Let me know if I missed any, or made any mistakes!

2 Likes

There are over a million known species of insects, and curators are supposed to check and repair them one by one. Greeeeeeeeeat.
So far I’ve done two genera, in something like a week.
Pretty obviously, we need a bot for this.

3 Likes

Today I repaired 47 of 48 species of Digrammia, the one European species had already been taken care of.

1 Like

Nearly all the taxa I fixed have a (current) 7 digit GBIF number starting with a 9. At least this aspect of the problem is nonrandom! I have to wonder whether indexing changes were made at GBIF, causing the links to break.
If this were to become a bug report, our admins need something structured enough to tackle. How could this be formulated to make it productive? It looks to me like GBIF involvement is needed (if my hunch is correct).

1 Like

Data being out of date is not a bug. It would be a bug if some programming in the code was adding the wrong identifiers.

The root cause is changes in taxonomy being made on either side, but mostly changes at gbif. Any bot or automated process to fix will have to constantly monitor the gbif database ( and likely thr whole database as it is unlikely any api has access to a change log ). That’s a hugely intensive process and one gbif likely would not approve of.

1 Like