'Needs ID' pile, and identifications

[fffffffff]

Use pisum’s url from tables at the start of this topic and add date parametrs in url.
https://jumear.github.io/stirfry/iNat_obs_counts_by_iconic_taxa.html

[lynnharper]

Thank you, but unfortunately I am so unskilled at using the API that I cannot figure out how to reach into the past, so to speak, and use it to figure out what the ratio of Needs ID observations to Verifiable was for a date in the past. I can use the Explore tab in iNat proper to figure out the current ratio for observations made before a certain date (and the ratio is less than 39%, for the couple of dates I checked), but I would expect that ratio to decline over time anyway, as identifiers get around to older observations.

Maybe we should ask @pisum to contribute their considerable expertise with the API to this question?

[dianastuder]

@ddubois2

a lot of observations of dubious quality

me too. But I pull up 10 obs at a time. If it’s the dubious, I glance thru, pick out what I can - and mark all as reviewed.
For the new, or few - I will try my best to move each ID on as far as I can.

[arboretum_amy]

@lynnharper

use it to figure out what the ratio of Needs ID observations to Verifiable was for a date in the past

Hmm, hard one. I’m not sure you can pull the past status of observations, only their current status? But I haven’t tried it yet. It might be easier to hunt down the links for previous Year in Review’s, although I don’t know how many years staff has been making those.

[lynnharper]

Good point. Hey, @tiwane, any chance you could weigh in here on how the ratio of Needs ID to Verifiable observations has changed over the years?

[arboretum_amy]

At least to me, it seems pretty stable at least over the last 5 years. I mean it does fluctuate by up to 6% but there’s no wild upward swing. Here’s from the Year in Review each year

2017, 36.7% Needs ID

2018, 40.18%

2019, 40.57%

2020, 37.74%

2021, 42.02%

Note: I am not completely sure whether these display observations that were Needs ID at the time or they are observations from that year that still need ID even now. I suspect it’s the latter.

[arboretum_amy]

It seems that 2014 is the first year for which the Year in Review link functions. Interestingly, unlike the last five years, Needs ID does seem to be lower if you go back more than 5 years ago, showing high 20s or low 30s rather than high 30s or low 40s:
2016, 32.89%
2015, 29.71%
2014, 28.63%

Edit: here’s a graph. However as I said above, I am not sure if these are observations from that year that still need ID even now or observations that were Needs ID at the time.

[pisum]

@lynnharper

Right now, 39% of the verifiable observations worldwide are Needs ID. What was the percentage five years ago? Or every year since iNat started?

@lynnharper

Similarly, how has the percentage of identifiers out of all observers/users changed? Currently, if we look only at verifiable observations, about 11.6% of the number of observers are identifiers.

it is possible to answer these kinds of questions only if you have a snapshot of a metric you’re trying to compare, from the point in time that you’re trying to compare against. for needs ID vs verifiable, i’m only aware of a few snapshots of data i’ve captured (ex. https://forum.inaturalist.org/t/needs-id-pile-and-identifications/26904/4).

iNat staff will have to say if they’re collecting more systematic snapshots of any of these measures.

…

with the year end / new year coming soon, it may be worth thinking about a few such metrics you’re particularly interested in comparing over time, and taking snapshots of those so that they can be referenced in the future.

[arboretum_amy]

@lynnharper

Similarly, how has the percentage of identifiers out of all observers/users changed? Currently, if we look only at verifiable observations, about 11.6% of the number of observers are identifiers.

https://www.inaturalist.org/stats/ will show that, but only for the past 12 mo:

The number of identifiers seems much more static than the number of observers.

[lynnharper]

Ah, thanks, @pisum! That’s what I thought, but I wasn’t sure. That’s why I pulled in @tiwane, because I assume iNat is keeping track of such things behind the scenes.

[lynnharper]

And thanks to you, too! If the number of identifiers has not grown at the same rate as observers over the past decade, then maybe the underlying issue is that we need to recruit more identifiers.

Perhaps I shall badger my botanist friends in the new year…

[arboretum_amy]

@lynnharper

If the number of identifiers has not grown at the same rate as observers over the past decade,

Did you conclude that based on what I said? I don’t think the info I provided supports that conclusion. The graph from https://www.inaturalist.org/stats/ is one year, not a decade, and the graph I made based on Year in Review numbers was just number of Needs ID observations, not referencing number of observers or number of identifiers.

[tiwane]

@lynnharper

That’s why I pulled in @tiwane, because I assume iNat is keeping track of such things behind the scenes.

As far as I know we haven’t done much delving into this, but I’m definitely not the stats guy.

[arboretum_amy]

@arboretum_amy

Perhaps it depends on your geographic location of choice? In my area there definitely are more Needs ID observations from students and new users that quit after 1-5 observations than there are from power users. But I could see how an area with less of a user base could be dominated by one zealous observer.

@ddubois2 You got me curious about this too so I’ve attempted some number crunching. My quoted statement above, although it feels true to me as I look through plant observations, probably isn’t true.

Observers in my county:
17,660 who have at least one observation in my county
13,830 who have at least one verifiable observation in the county
(Therefore I assume 3,830 people have only casual observations.)
8,764 who have at least one observation in the county that Needs ID.
(Therefore 5,066 people have only research grade observations in this county, wow.)

Observations in the county:
317,785 total
277,201 verifiable
73,684 Needs ID

The top 500 observers of verifiable observations individually range from 59 verifiable observations to 17,603. Collectively they have 192,756 verifiable observations or 69.5% of total verifiable observations.

The top 500 observers of Needs ID observations, which are not necessarily the same people as above (259 overlap but the rest don’t) individually range from 18 Needs ID obs to 4665. Collectively they have 46,802 or 63.5% of the Needs ID observations.

If I consider only the 259 people who are on both lists, they have 63.7% of verifiable observations and 59.0% of Needs ID observations.

[lynnharper]

Nope, I agree that what you provided from https://www.inaturalist.org/stats/was limited to one year, and that the Year in Review numbers was just the number of Needs ID observations.

What I did conclude from the one-year stats was that it was possible that the ratio of identifiers to observers has decreased over the years.

[lynnharper]

So who should we ask for those sorts of stats?

[arboretum_amy]

It’s certainly interesting how the observer:identifier ratio fluctuates from about 2:1 to 5:1 just depending on the time of year.

[pisum]

@lynnharper

Similarly, how has the percentage of identifiers out of all observers/users changed? Currently, if we look only at verifiable observations, about 11.6% of the number of observers are identifiers.

i thought a bit more about whether this can be answered without snapshots of earlier data. it won’t reflect things exactly right at past points in time, but you can estimate data from past points in time using some API calls. here are the results (for verifiable observations), as retrieved on 2021-12-28:

i’ll let you interpret the data for yourselves, though i will caution against using this data alone as the basis for saying whether or not growth in identifiers is sufficient.

…

in case anyone wants to try to replicate or re-run my numbers above, i created this in Excel. assuming you have Excel and your version / setup allows you to run the LAMBA and WEBSERVICE functions, you could create the same thing by creating these custom named functions (go to Formulas > Defined Names > Define Name…):

• getCountFromJSON
  get the numeric value associated with the first element (total count) in the API response (in JSON format)
  =LAMBDA(j,MID(j,FIND(":",j)+1,FIND(",",j)-FIND(":",j)-1))
• getObserverCount
  Get number of Observers based on verifiable observations created up to a given datetime
  =LAMBDA(d,getCountFromJSON(WEBSERVICE("https://api.inaturalist.org/v1/observations/observers?per_page=0&verifiable=true&created_d2="&d)))
• getIdentifierCount1
  Get number of Identifiers based on identifications up to a given datetime (for others’ verifiable observations)
  =LAMBDA(d,getCountFromJSON(WEBSERVICE("https://api.inaturalist.org/v1/identifications/identifiers?per_page=1&quality_grade=research,needs_id&current=any&own_observation=false&d2="&d)))
• getIdentifierCount2
  Get number of Identifiers based on identifications up to a given datetime (for verifiable observations)
  =LAMBDA(d,getCountFromJSON(WEBSERVICE("https://api.inaturalist.org/v1/identifications/identifiers?per_page=1&quality_grade=research,needs_id&current=any&d2="&d)))

[fffffffff]

It’s understandable with summer urge of observing and stable iders who work year around.

[spins]

For me, the big turn off (concerning ID’ing plants) are the way too many observations of garden plants, pot plants, plants in botanical gardens etc. Why people think what they grow in their garden is relevant for Inaturalist (and similar platforms) is beyond me.