Standardize and Clean Up Observation Fields

Observation Fields are a potentially very powerful tool, and do get a lot of use, but unfortunately because anyone can create a field they are plagued with many repetitive or redundant or contradictory options. We often direct people to fields when they have feature requests on the forums or site, but the problem is the fields themselves need a lot of work.

image

For common subjects of a field, such as number of organisms, there are pages and pages of these…

Here are a few ideas below though others may have other ideas:

  • automatically delete any field that was made a long time ago and has no entries.

  • Create curator tools that allow merging of fields without any loss of observation data, project affiliation, etc

  • Begin an effort, perhaps with specially created tool, to merge and standardize the most important fields. Create a system where there are ‘curated fields’ which are semi-official and managed by curators, and encourage people to use those or merge into those. make the curated fields prominent on the Fields page.

  • Consider making it so that only curators - or even a subset of curators - can create new fields. Make sure curators search for an existing one before creating a new one, just like with taxa.

  • Offer ‘suggested fields’ based on organism type, or else just offer a few commonly used ones, easy to find within the observation. Options might be number observed, alive/dead, habitat/natural community type, leaf phenology if a plant, etc. Don’t make them mandatory or ‘naggy’ but make them easy to find.

  • Eliminate required fields from projects, instead create suggested fields and have them auto populate on an observation when you join the project and create a link from the project to the Identify functionality, filtered for that project, so people can add them that way.

  • Allow user to set a few default fields to automatically populate (empty of course) in all their observations so they don’t have to search for them.

  • Improve searching with fields. For instance I can get a list of taxa that occur in the ‘Poor Fen’ natural community by abundance - very useful - by searching with that field. However, i can not get a list of natural communities that Carex stricta occurs in using the same field.

Any other thoughts?

Agree across the board, the only change / addition I would make is that it would be great if the curated fields suggested mirrored the terms and naming standards used by the DarwinCore framework.

Oh and some process needs to be implemented for translation, so the same field can be used across multiple languages, but still retain the same unique distinct field ID.

3 Likes

Maybe, if fields are going to be very well curated, i.e. approved within hours, and not potentially going months or years before curation like some other pieces of the site… One of the main reasons iNaturalist can be a useful tool for research projects and other data-collecting projects is because they can create the exact extra data fields and drop-downs needed for their purposes. That might mean the options for the field are tweaked slightly from the “standard” version of the field, e.g. for “Flowering Phenology” the project may require being far more detailed than simply budding/flowering/fruiting. Even if well-curated, I can still see this being an annoying barrier to entry.

I don’t think this feature should be eliminated. Some will want to maintain the extra fields must be filled out for addition to the project.

1 Like

Maybe something somewhere between the two ? Continuing to allow all users to create new fields at will seems to be a recipe for simply getting back to where we are now even if a clean-up is done.

Maybe requiring a curator approval to activate a new field ? Allowing anyone to create one, but it only becomes available for use after a curator has reviewed it to ensure it can’t be replicated via an existing one, or even communicated with the requesting user to see if they know of or can use the existing one.

I get the concern about slow curation, etc… but i don’t see any other way to stop this flood of duplicative fields. And unfortunately as it stands now, the fields are very limited in their use, especially to newbies, because of that. Another alternative would be to let anyone create whatever, but also create a subset of ‘curated fields’ including a standard one for number of organisms, dead/alive, and those other commonly used ones. But without tools to merge, they might just never get used anyway.

The required fields absolutely break projects imho, making it a huge pain to add things to them. I outright won’t even participate in projects that have required fields. I guess if a lot of people really need required fields for some reason, maybe it’s a bad idea to remove them… but seriously, i don’t see any case where some observations without fields filled out are worse than many observations just never being added at all… and after all the project proponent can use the identify page to add them all in fast anyway. I guess a hybrid approach is just having collection projects that filter based on the fields, that is the best of both worlds i suppose, except that you can’t see obscured locations.

Of course, i’d be happy with just some of these changes being implemented too. Better than nothing. As it stands now i have a few standby fields i use, but i use fields a lot less than i would otherwise because they are such a mess.

I’d like to use fields more and this sounds like a good general set of changes that could make it 1000% easier. As it stands, I’m only confident using breeding bird codes (which I found by accident) because I am already familiar with them as they’re standardized and not unique to the site. I am usually only able to find what I need for other taxa by guessing at what they might be called and then having to pick the wrong thing a few times before either locating what I think is the right field or giving up and not using a field at all. Defaulting to my “favorite” fields would rock…making it easy to search for fields/ have suggestions based on taxa would also rock and I know that personally I would make a point to add a lot more data to my observations if this was easier to do.

1 Like

Relatedly I created a forum wiki where we can start looking at choosing standardized versions of fields and tracking some useful ones. Should be helpful whether or not these broader changes occur. Feel free to add to and edit it. https://forum.inaturalist.org/t/observation-field-standardization-wiki/380

  1. automatically delete any field that was made a long time ago and has no entries. THERE ARE FAR TOO MANY

  2. Create curator tools that allow merging of fields without any loss of observation data, project affiliation, etc
    I think you will need this that few that it can be done by the staff…Making tools takes also a lot off erfort i think. Merging is a good idea anyway

  3. Begin an effort, perhaps with specially created tool, to merge and standardize the most important fields. Create a system where there are ‘curated fields’ (VOTED)which are semi-official and managed by curators, and encourage people to use those or merge into those. make the curated fields prominent on the Fields page. If most of my observation fields are included i think it is an excellent idea. In general i only uses 2-4 fields often i think.

  4. Allow (PROPOSE) user to set a few default fields to automatically populate (empty of course) in all their observations so they don’t have to search I thought that a set is proposed in front and that a user has to switch them off or put them in an ADVANCED tab.

  5. Oh and some process needs to be implemented for translation, so the same field can be used across multiple languages, but still retain the same unique distinct field ID. You have a FIELD and a FIELD has serveral LOOKUP VALUES…and these values can be in ENG, GER, SPA or italian. I think one should include the display text per language of field (optina Number of female, Number of Young, Number Male, Number eggs)

  6. TAGS. Is it not possible to solve al other problems with optional tags ? One can provide a library of tags (for example for biotoop) but a user is free to add personal or project TAGS ?

I have problems with the editor…but it is not possile to add files ? Or only .jpg, png, gif. It is not possible to add (HTML) tables ?
I would like to have these fields included, but if you with i could reduce the fields rather easy with the most important fields. I still do not under stand the fields used in iNaturalist and i think there are far too many which makes it difficult to use. One can have one field and ad DIFFERENT lookuptables to it and even translate these different lookuptables. And i would also love if some fields could be used in the iphone app, that would be convenient.

Lookup table for different taxonomy groups, one language
https://www.inaturalist.org/journal/ahospers/21912-lookup-tables-related-depending-changes-with-taxonomy-group

I am still not understanding what this huge list is. Is this a list of VALUES you would like to have in one field? Or would you make a field for every one of those? I don’t think the latter is a good idea.

These are the fields i use.
There are about 7 fields
There are about 12 groups(birds, mammals, fish, plants) and the context/List of values can vary within the group so the amount of options are 12*7=84
I never under stood the fields iNat uses so i hope to figure out what is the best way to go, call it mapping ??

I will remove some and add pictures if the editor allows me Maybe that is more clear. My original proposal was to use one table with all drop down values for all groups with different values for each group. And i included the table with values.

Lookup table for different taxonomy groups, one language
https://www.inaturalist.org/journal/ahospers/21912-lookup-tables-related-depending-changes-with-taxonomy-group

Well it definitely kinda breaks this post, can you edit it out and link to it instead maybe?
If I am understanding you right you could make those fields on inat but might be a pain to use the drop down with so many options

  1. make those fields on inat but might be a pain to
    I just do not want to add another field which probably already exists on iNaturalist with a few cosmetic changes. But this seems to be the way to go in Inaturalist ? And second i would like to use the default field
    The value of the field changes by taxonomy. If you have FISH than you see other values in the field than if you have a bird.
    I must agree that first time users agree that it is a bit hard (but so many user fields are also very hard) but hard core users love it because it covers 99% of the needs for 100.000.000 observations and 40.000 users.
    The main complaints are for collectors it thought…so if you have determinator, a collector, a collection and a first and second determination by several people.

The same field ‘appearance’ (plumage) for Birds (=1)
image

The same field ‘appearance’ (plumage)for fish(=9):
image

Lookup table for different taxonomy groups, one language
https://www.inaturalist.org/journal/ahospers/21912-lookup-tables-related-depending-changes-with-taxonomy-group

The same field ‘appearance’ (plumage) for Plants(=8/10):
image

Lookup table for different taxonomy groups, one language
https://www.inaturalist.org/journal/ahospers/21912-lookup-tables-related-depending-changes-with-taxonomy-group

ok, i am starting to understand that. i think having fields link to individual taxa types makes sense for some of them.

it’s a hard situation. cassi brings up the valid point that making fields curator-only may just create more work for already overburdened curators. TBH that still sounds better to me than the current situation, but I understand the concern. I think a hybrid of some ‘suggested’ or ‘curated’ fields and then letting the wild west of user created fields exist too, might work

You can not just offer a default group of fields which can be easily (if it is a wish, i have no idea why you want this, but it seems to difficult for normal people) removed if it is a wish ?
Or there so many special wishes for the fields for the projects ? With the current situation i will not use the fields because there are too many

i can’t. I offered that as one of the suggestions in this thread, but only admin can do such a thing. So i guess… vote for it or if you think it’s different make another request?

1 Like

OK, I know this is not the answer you wanted to hear, but we talked it over and Observation Fields were designed to be a place that iNaturalist doesn’t exert much influence over. It’s user-controlled and we’re not going to step in here as far as organizing and standardizing things.

So I’m going to close this, but note that:

  • I’ve submitted a GitHub issue to make observation field search results be ranked by the number of observations a field has been used with, to at least bring up the more often-used fields from a list of fields with duplicative functions. I’m hoping this will cause some to be more used than others.

  • I’ve also posted an Annotations topic in General as a way to get some more standardized metadata fields available for certain taxa. I know this is something we dropped the ball on in 2017.

  • I’ll add your requests for better Obs Field search options to our team document for our planned upgrade of Search.

3 Likes