Searching for Annotations - Basic to Advanced

Advanced Annotation Filtering

The Basics

The Fun Stuff

The above information will cover the majority of use cases for almost any users. But sometimes you want to filter things down just a bit more. The main limitation of the search URL functionality using the annotation options is the inability to filter on multiple different annotations. You can choose to include or exclude multiple values, but only within the same annotation. With a combination of annotation options and project options (both project_id= and not_in_project=) I made a feature request that was accepted, but it sounds like a lot would need to change for this to get implemented. So, as a workaround, I made projects for all different annotation values. I'm hoping one day for the search URLs functionality to be upgraded so all these projects aren't even necessary. Here is a link to the parent collection project. All the child projects can be found there (click "Read More" under the About section).

Annotation Projects

Examples

Observations with Life Stage - Adult and Sex - Female

https://www.inaturalist.org/observations/identify?quality_grade=needs_id%2Cresearch&term_id=1&term_value_id=2&project_id=225556

Observations of Lepidoptera without any Life Stage annotation and without Evidence of Presence - Leafmine

https://www.inaturalist.org/observations/identify?without_term_id=1&not_in_project=evidence-of-presence-leafmine&taxon_id=47157&place_id=any

Observations from the Antarctic Penninsula without ANY annotations

https://www.inaturalist.org/observations/identify?quality_grade=needs_id%2Cresearch&place_id=120492&not_in_project=annotation-projects

Observations with annotations for (Flowers OR Flower Buds) AND Fruits/Seeds

https://www.inaturalist.org/observations/identify?quality_grade=needs_id%2Cresearch&project_id=flowers-and-fruits-flower-buds,flowers-and-fruits-flowers&term_id=12&term_value_id=14

Discussion

As you can see, you can get quite detailed and complex with these search URLs. The main benefit of also using projects to search for annotations is the ability to search on multiple different annotations. But the hidden, less obvious perk is to be able to search with AND and OR. Comma separating projects in the url (both for project_id= and for not_in_project=) treats the search like an OR. The same goes for term_value_id=. So you can search for observations with Organisms or Tracks. But when you combine the two as I did in the last example, you can include only observations that meet both the project and annotation filter criteria. Unfortunately, since comma separating the urls treats it like an OR clause, we are still limited to two values that we can include or filter out. If the apply_project_rules_for filter worked as expected for annotation values (see this bug report), we would be able to bump that number up to three.

Another benefit I've discovered is the better handling of excluding an annotation. As you can see in the Lepidoptera example, it filters out only observations that have the leafmine annotation. If you were to try to replicate this with the annotation URL filters using without_term_id=22&without_term_value_id=32 (which cannot be configued with the UI), it will ignore the value_id. If you do term_id=22&without_term_value_id=32, it will filter out leafmine annotations, but only if the Evidence of Presence is filled with SOMETHING. So this is the only way to truly exclude what you want.

I wasn't sure if a tutorial or journal post was more appropriate, so I just copied it in my journal.

If you have any other use cases that I may not have covered here or if something I wrote doesn't make sense, leave a comment and maybe we can figure them out together.

16 Likes

You can use https://www.inaturalist.org/observations?taxon_id=47157&term_id_or_unknown=22&without_term_value_id=32 to return leps excluding EoP leafmine, while retaining those with no EoP.

1 Like

Thanks, that’s good to know. I didn’t know about that functionality until your post here

https://forum.inaturalist.org/t/identify-filter-without-annotation-not-working-properly/30148/24

I can try to update the original post. But I think having the projects available as an option still makes certain things possible that aren’t with the current URL filters. And sometimes it’s simpler to build too, at least from my perspective.

1 Like

I think no one on the forum realized that functionality existed until then :sweat_smile:

At any rate, I agree the projects are a good option, I just wanted to mention the other parameter as something that’s available.

1 Like

this issue was considered in API repo but not solved.

I stumbled upon this trick in your journal especially when looking to search for things “eggs”, people who annotate and search predominantly pick and URL filter only with life-stage : egg thing.

But there exists very common alternate annotation that if not popular, atleast is more powerful, Evidence of Presence: Egg. It is powerful because EoP allows multiple annotations - so one can annotate both Egg and Organism in it to provide a powerful combined annotation of Parent-egg laying observations directly which offers more proof for IDers who are searching or IDing them than eggs alone. And I have also seen people pick arbitrary adult or egg for LifeStage annotations in those observations because either is correct technically aka your URL filter on LS: eggs will always exclude some correct observations with eggs which were not just using LS but relying on EoP annotations and correctly too.

so until the issue of filtering multiple annotations is solved in above github request, filtering via your projects trick is the only “easy” way. I created project for LifeStage:eggs and then filter via evidence of presence eggs annotation to look these alternate set.

1 Like

Ah yea I use this on observations where a female mantis is laying eggs, those observations are very crucial as they help confirm which ootheca belongs to which species. that way it doesnt get dumped together with all the regular oothecae observations and you can filter them

I use evidence of presence: egg annotation for my project so that its easier to pick out observations with nymphs hatching out of the ootheca/ female laying eggs/ simply eggs from the observations

Just for interest’s sake - I wrote a journal post on the Power of Annotations

Also, you can use @wy_bio ‘s alternative Explore and Identify pages to use clickable filters for annotations - iNaturalist Explorer

1 Like