Thanks, Dave :-)
I personally would still prefer bulk edit and add features rather than letting AI loose on it. Perhaps I’m just old fashioned? Or maybe just old ;-D
2 Likes
I dunno… it’s much less “work” to just be correcting a few AI mistakes than having to do the whole lot manually. Even if they are checked in their entirety it would still be approx half the workload… but I do question whether we NEED it done, or just IT WOULD BE NICE TO HAVE IT DONE. Maybe if they come up with some onboarding tutorial thing like they do with the forums, they could have the annotations part of that taking the new users through how to apply annotations to existing observations, with the lepidoptera being an obvious easy training case. Each new user gets to do 5 lep annotations each before moving on to the next task… could also ingrain a practise of applying the annotations on new observations, reducing the future workload of this!
If you made the project a collection project, then all you would have to do is annotate observations. They would then automatically be included in the collection project. Or am I missing something?
2 Likes
Tony - I see two issues with this. There is the ongoing challenge of accessing details of obscured observations in collection projects, so that doesn’t work for many people (speaking as a data user).
The second issue also addresses karoopixie’s comment. The math is against manual annotations and I don’t share your belief that waiting for people to manually annotate millions of photos is realistic. I’ve posted elsewhere that of the over 5+ million insect observations on iNat, only 20% have annotations. There is no evidence that it will get better. That obviously excludes the other possible annotation categories, but I can’t imagine things improving much.
3 Likes
Tony, yes your suggestion would work if all obs could be annotated. Unfortunately, the rearing obs which have all life stages cannot be annotated, and these are the most valuable ones for identification because it has the larva and adult confirmed.
And, yes, I am well aware that iNat only wants one individual in one life stage per ob. But, hundreds of rearing obs were migrated from iSpot, and the posters are not going to split them all up again. Some still prefer to upload rearings as a single observation, and I can’t say I blame them. It is a nightmare trying to link all the separate obs for one rearing together.
I am thinking… Let me do a little experiment, and I’ll get back to you with the results…
1 Like
Okay, I have done my little experiment.
I created a collection project for the same params as the trad project. These are the current totals:
Numbers Tradpro Collpro
Observations 4505 3794
Species 387 312
This is mostly because people automatically add to the project without adding any annotations, and because the rearings are not annotated.
The projects are here if anyone feels inclined to see them:
Trad project - https://www.inaturalist.org/projects/pillar-parade-s-afr
Coll project - https://www.inaturalist.org/projects/pillar-parade-collection-test
I will be deleting the collection project soon as it was just done as a test. You will also see anomalies in numbers of obs outside the southern Africa place because people have randomly added those on the trad project, but all are added on the collection project (and I wouldn’t necessarily include all those places on a coll project since it was supposed to be just for southern Africa anyway but I included them for testing).
3 Likes
I hear you. We were talking about your proposal yesterday and it is possible, although it would take some time to train. But as I said in that thread, just as our species ID AI was trained on observations with millions of IDs made manually, so would this AI, and the community would still have to go through and correct AI mistakes for the next training run to improve things. As to whether we have “enough” annotated observations now to train an acceptable model, that’s something we can look into. But more will pretty much always be better.
@karoopixie interesting experiment, thanks for sharing.
3 Likes
You have a trad project, and now a collection project. Create an umbrella project that picks up both, and then (after waiting enough time for indexes to catch up) do a search on leps NOT in the umbrella project. That will be your shortlist of what NEEDS to have annotations added. As soon as you add the annotations, they will appear in the collection project, and so will drop out of the umbrella project…
I’ve not tested it for annotations, but we do a similar thing to catch obscured records into our bioblitzes…
1 Like
Thanks for your suggestion. I had considered an umbrella project (though in a different way) but it does not solve the initial problem, and creates many projects.
I already have a way of finding the unannotated leps which is much simpler than your solution (which is not to say your way doesn’t work, and it may have applications elsewhere), which is to do a filter on the Identify page like so:
All sA leps: https://www.inaturalist.org/observations/identify?reviewed=any&quality_grade=needs_id%2Cresearch&order=asc&taxon_id=47157&without_term_id=1
Just sA moths: https://www.inaturalist.org/observations/identify?reviewed=any&quality_grade=needs_id%2Cresearch&order=asc&taxon_id=47157&without_term_id=1&without_taxon_id=47224%2C47653
I work on the second filter because as you can see I only have 8,000+ (of which about 400 are unannotatable) moths to annotate. I’ll leave the butterflies to someone else because I simply don’t have time or inclination to annotate 18,000+ obs (I have already annotated ~37,000 obs).
Which leads us neatly back to the original feature request of bulk adding annotations, and bulk adding to projects. This would greatly streamline the whole process, and eliminate the need to create multiple projects to get to the same destination.
2 Likes
Cool… add the “not in project” filter to yours, and it might cut down the list a bit.
Ah, no, you see that is Step 2 of the process!
See my comment above:
The link for Step 2 is:
Larva obs not in project
that is not what I mean… you have a filter that gives you all the un-annotated leps, but some of those might already have been added to the target project manually. As you say, there are a lot of observers and identifiers that add larva obs to your project directly. The addition of the “not in project” filter is just so you don’t have to work through those when you are putting the annotations. Unless of course thoroughness is important, and they must ALL have annotations!
I can see why creating multiple projects is a pain, especially when you have a base of followers that are usually accessing the project through one project page and they would likely have to start looking at another (the umbrella project), but that doesn’t make your way much simpler than mine. Your step of then having to go through and manually add them to the project would be completely eliminated by the collection project being in play. If you had 50% larva obs, then that would be 25% off your total workload. How is that not easier? It sounds very much like you are quite attached to your process!
But yes, bulk add and bulk annotate would definitely be easier, and wouldn’t require the extra projects.
I’m in NZ so it won’t affect yours, but I do try and add life stage (and sex where determinable) for lep obs in NZ.
There was a topic started recently, where you could nominate a taxa, and give tips etc on how to ID, and then volunteers could join you in working on those IDs. This would be along those lines, you could post the filters and have volunteers doing the annotation side of it. Can’t think how to find that topic at the moment though…
[edit] Found it!
https://forum.inaturalist.org/t/help-me-identify-non-experts-welcome/2915
1 Like
Thanks for pointing out that I may be attached to my process - we all need a kick in the butt now and then ;-D
I created an umbrella project just to see what we’ve got so far:
https://www.inaturalist.org/projects/pillar-parade-umbrella-test?tab=species
I don’t find this particularly helpful (but then it is afternoon and I haven’t had lunch yet, so it may look more rewarding in the morning).
Yes, I do want all annotations added because I think the graphs on the taxon pages are very useful. I just quickly got these 2, which are maybe not really representative of what I want to show, but they’re interesting nonetheless:
Hippotion celerio
Hippotion eson
This is fantastic data which can tell us flight periods, relative longevity of life stages, number of generations in a year, differences between similar species etc. So, yes, maybe I’m a bit OCD about adding ALL life stages ;-)
Anyway, I think if the following three features are added, especially the first one, it will go a long way to making the whole process faster which will encourage more people to add annotations/projects and take the load off the 3 or 4 who are doing it now (for this particular project):
- Add projects to annotations tab of identify modal
- Bulk selection of observations to add to a project (this thread)
-
Batch adding annotations
And - AI to add annotations
Thank you for adding annotations to the NZ leps - I know what a big job it is, but the rewards are very great.
I will have a think about asking for annotaters on the other thread you linked (after lunch).
I do appreciate your help and suggestions very much, even if I don’t necessarily follow them. They always do send me down an interesting road though, which is fun 
2 Likes
I totally get the CDO thing… :)
I’d like to add all the observation of this project (just used in New Zealand) to this project (international). Currently that’s really inefficient because the only way to do it is to view the observations in the one that aren’t in the other and open them all in new tabs and add them one by one.
This particular case could be improved in a few ways. Probably the best would be
using an observation field automatically with a collection project (a field like this one could be added to all of them relatively quickly on the Identify page), but without that these would help a lot
1 Like
I have been invited here by @mercantour to add my observations made in the Alpes-Maritimes into the project Flore des Alpes Maritimes.
After filtering out observations made in [botanical] gardens, I get 315 observations to add to the projet.
This discussion confirms that there is no easy way to do it now.
I might develop a Windows console application to do it, based on the POST /project_observations API. The tool inputs would be simply the project identifier and a search URL for grabbing the observations to add to the project.
1 Like
With the help of @mercantour, I found how to add a batch of observations (resulting from a search URL) to a project.
Below is a tutorial for an observer wanting to add many observations to a project.
Join the project:
Filter the observations to add:
In this case, the search URL to get the observations to add is:
Rewrite the URL as follows (replacing “observations?user_id=jeanphilippeb” by “observations/jeanphilippeb?preferences[per_page]=200”), applying the same filters:
This URL opens a page that enables editing 200 observations.
Click on “Batch edit”.
Click on Select “All”.
Click on “Add to Project”.
Click on “Add”.
Wait for the process to complete.
Click on “Next” to treat the other observations.
5 Likes
I want to bulk add because I have a lot of technologically challenged members. It would be so much easier to do bulk adds myself than to try to teach them all how to do it. Also a lot if times I add obs from people who are not in the project. Sometimes they are fly-by-night users. I may never get in contact with them, but they may have 20-25 obs I want to add. It becomes very tedious adding them one by one when I have to load every observation page with my slow connection.
2 Likes
So, maybe I ought to make this console application mentioned above.
3 Likes