Lots of things to consider. Here’s one more example. Others can try to do others if they want. In this one I’ve used white to indicated multiple kingdoms rather than just green for plants in the first example. This example has obscured observations everywhere. So, I’ve put a grid everywhere, except I appear to have missed the top right corner.
Opacity is a problem if you want it to represent amount of observations using it. I just used one opacity. Make it too opaque because you have a lot of observations and you have the same problem as it being obscured by a lot of circles. In this example, you lose the fact that two of the rectangles have most of the obscured observations. That said, maybe a few levels of opacity that aren’t to opaque or transparent would work. Here’s a version with more opaque rectangles (50% for dense and 20% for the rest).
I think this would still work good with the density grid style markers. When zoomed out far enough that the rectangles are smaller that the density grid markers, those markers would take over.
Let’s say I zoom in past the borders of the rectangles though. Now what? I lose the rectangular representation of what is going on and maybe everything is just a hazy red, blue, green, or white.
Bottomline, it may be a good idea and everything should certainly be well considered before implementing.
Also, it seems like it would be really easy to add a way to turn off obscured observations using a search filter. The “threatened” filter partially does that already. If one of the big problems is map clutter, just add a switch to turn off part of that clutter (default of obscured observations visible). It doesn’t fix everything but it is a good start and doesn’t hurt anything.
easy depends on who has to do the work. besides handling a new obscure grid map style and associated interaction functionality, the actual pieces needed to split off obscured observations into a separate layer that could be toggled on and off would be something like:
change the API to incorporate a new parameter that functions an OR with the existing geoprivacy and taxon_geoprivacy parameters. (currently, you can filter for geoprivacy=open&taxon_geoprvacy=open to get non-obscured observations, but you can’t filter for cases were geoprivacy=obscured OR taxon_geoprivacy=obscured.)
make a decision whether you want to handle the 2 types of obscured observations independently (as separate layers) or lumped together (as a single layer).
in the explore-style pages on the website, instead of displaying just a basemap + a single observation overlay + associated UTFGrid layer (for interaction with markers):
a. split the marker overlays into obscured and non-obscured layers
b. split the UTFGrid (interaction) layers into obscured and non-obscured layers
c. add a control(s) to toggle display (or opacity) of the overlay + UTFGrid pairs
d. reformulate the map legend to account for the various changes.
for the taxon-page-style maps (in the taxon page, observation detail page, suggestions section of the Identify page, etc.), first decide whether you want obscured observations to include casual observations or not. if you include them, do you include them all or do you have separate categories that correspond to the different layers / categories of casual observations that those maps display? split those maps accordingly, and update the options in the layer selection control accordingly.
the observation detail page currently displays the obscuration box if the given observation is obscured. so you would probably remove that in favor of just displaying the obscured observation layer(s).
account for other variations of the taxon-page-style maps. for example, the batch edit page uses a map that’s similar to the taxon-page-style maps, but instead of verifiable and various categories of casual, they split out observations into all and featured. so you’d have to do all+non-obscured, all+obscured, featured+non-obscured, featured+obscured.
there’s a special category of taxon-page-style maps that use a heatmap style as opposed to pins and density grids. decide how you want to handle those?
for the various other one-off tools that have various maps (side-by-side comparisons, time series maps), decide whether you want to incorporate handling for the squares + display toggling. (i would assume the iNat staff would not do this since these are one-offs and built using a different set of packages than the cored iNat maps.)
reformulate the Explore pages in the mobile apps, using the same thought process as #3 above.
(possibly) scale up the processing power needed to handle multiple times the load needed to render and deliver the maps.
write up something that describes the changes, in multiple languages, and disseminate that to the user base, the network affiliates, etc.
over time change up tutorials and other materials to incorporate the new changes.
compare a more incremental change, such as changing the style of the markers (ex. make the colors of the circle markers more transparent without making the borders transparent, which would deemphasize the markers while still making them possible to locate):
design new markers
deploy new markers to the map generator
switch out new markers in the Explore page key
(you could – but probably would not need to – update tutorials and other related materials and educate the masses about the changes)
i think the last part of this is really the key. the devil is in the details for this kind of thing.
I’m not sure this would help. There are two conflicting issues at play here. When there are too many markers, you can’t see what is under them. This wouldn’t solve that as too many borders would still be a mess. The other problems is that when there aren’t a lot of markers, they are already really hard to see due to the transparency, especially the green ones but likely others depending on the background color. They could be easier to see if the border were not transparent but that again makes it more problematic when there are a lot of them.
I thought a little bit more about the problem of when you are zoomed in closer than the rectangle borders (you are viewing inside of a rectangle) and have decided that the obscured observations could just disappear when you zoom in that close as the accuracy of them would be so poor as to not really have any value in displaying them. That would also help with the probably of people interpreting them as accurate. In the genus I’m studying, there has been more than one comment on an observation where someone went out to do surveys for a plant and collect more data only to not find any and then I had to tell them that the point they went to was purposefully changed to be in a random location. They could have paid more attention and noted that it was obscured in the observation, but in some cases people just search for a species, look at the map, and then go.
this can all be tweaked. the opacity of the marker colors can be adjusted, the opacity of the marker borders can be adjusted, the design of the borders (maybe using a double white+black border for contrast) can be adjusted, the threshold at which markers get merged together (to reduce number of markers and to reduce overlapping) can be adjusted.
i’m 99% sure there is some mix of all of these variables that will produce a decent visualization that will make individual points easy enough to find while still deemphasizing obscured points as a whole. and as much time as it would take to even just prototype a starting visualization design for a hypothetical obscuration grid map style, it would take a small fraction of that time to get a good alternative design for the existing obscured pin markers.
this seems wrong to me. as much as some folks seem to dislike displaying the obscured observations as points, it seems like excluding them from visualization altogether would be the exact opposite sin.
that’s part of why i’m not a huge fan of separating the obscured observations out as a separate layer that can be toggled on and off. for example, in the map of mountain goat observations below, you can see that all observations in Washington and Idaho are obscured, but the ones in Montana and Oregon are not. if you were examining activity along the border and zoomed in beyond whatever threshold you propose would cause the obscured observations to disappear from the map, that could easily cause folks who weren’t paying attention to think that there was no mountain goat activity in Washington and Idaho. (to me, that’s no better than someone who’s not paying attention thinking there is activity at a specific spot, not taking into account positional accuracy.)
That might work ok if the circles and pins displayed for a couple further out zooms. Worst case scenario is the zoom level where you first see the grid but obscured observations are often in the wrong grid square which to me is worse than the circles zoomed further in where at least experienced users know the points are obscured. The map becomes useless at the closest grid zoom if there’s more than just one or two obscured observations. But I suspect that has something to do with website loading efficiency
this issue is just related to the tradeoffs that each map style makes. the grid style (red squares) display a (pseudo-)density with at maximum of 32x32 markers per tile (which limits the maximum amount of rendering/processing needed). on the other hand, the pins/circles display the iconic taxon (color), obscured status (pin vs circle), and research grade (extra center dot) for the latest observation at roughly that spot, though it won’t account for other (older) observations at roughly the same spot. no single visualization is going to be able to satisfy all possible visualization desires, which is why i think the best thing is to focus on being able to toggle between styles and to incrementally improve each style, as needed.
i’m not generally opposed to a 0.2 x 0.2 grid visualization, except when its intent is to segregate obscured observations. (as i noted, i think it makes things too complex, both visually and technically.) but really, i think a better type of visualization than that would be something like a modified heatmap that incorporates positional accuracy into the way it renders its levels. that would allow both obscured observations (with roughly ~20km accuracy) and non-obscured observations with positional accuracy values to be visualized using the same set of rules. the only issue with this kind of visualization style is that while the pins and grid are good for interaction, heatmaps aren’t as good for interaction because everything blurs together by design rather than being individual clickable points. so that’s the downside of that kind of visualization, though i’m not against that kind of visualization being implemented without interactivity functionality. i don’t know that this would necessarily fix the visual noisiness issue (which you noted was your primary concern), since it would be noisy by design, but it would address concerns related to the accuracy of coordinates (which was another concern you noted and which seems to be a primary concern for others).
I would only agree they should automatically disappear when zoomed to a level within one of the rectangles.
At this zoom, showing obscured observations as rectangles or something makes a lot of sense:
When I zoom in much further as in the following image, displaying the obscured points becomes meaningless and increases confusion. The error radius on the points far exceeds what is actually shown on the map.
When you zoom way out as in your example of half of the United States, that is where obscured points become very meaningful and should definitely be shown one way or another. Their location is fairly accurate when zoomed that far out.
right. i understand what you’re describing, but i think it’s just strange to see data at one zoom level and then have it disappear when you zoom in. for example, in my mountain goat example, suppose you’re looking at observations near Portland, Oregon, at zoom level 10. you’ll see a non-obscured casual observation in Portland and several obscured observations across the border in Washington (or, if the 0.2 x 0.2 deg grid box map style is implemented, a box or 2 across border in Washington). and then when you zoom in, you expect to see the Washington pins or boxes to still be there in view, but then they disappear completely, while the Portland pin remains. to me, that just seems odd. if i’m seeing that happen for the first time, it would definitely take me some investigation to figure out what’s going on there.
when i first saw the obscured observation circle markers on the map, i had to think about those for a moment, too, but there’s at least a legend on the map that describes that the circles represent obscured observations. i’m not sure how you would represent the disappearance of data on a legend.
it’s true that some non-obscured observations are excluded from mapping because their positional accuracy values exceed the length of the diagonal of its 0.2 deg x 0.2 deg box, but they are excluded at all zoom levels. they don’t just disappear when i zoom in, and i typically won’t encounter a whole group of unmapped observations either. (typically the “unmappable” observations are one-offs.)
it’s also worth noting that there’s not a specific zoom level where the obscure boxes exceed the size of the map. it just depends on the screen/browser resolution/zoom, the size of the map window, the latitude, and the map zoom. so most likely you’d have to just pick an arbitrary map zoom level as the threshold for displaying / not displaying the obscured observations.
i mean i guess it comes down to whether you care about fine scale mapping or not. Problem is, the visualizations desired by the people who do not care, make it not really work for those who do care, but apparently some who don’t care don’t want to change them because it’s ‘too hard’ or ‘ugly’ or something.
And yes removing or denoting observations with really poor locational uncertainty (or none noted) would be really valuable too, its just different from this request
i think what you’re saying here is approaching what i’ve been saying. i’ll try one more attempt at clarification of this idea with an analogy…
suppose i’m an owner of a burger restaurant. i see that a small but significant portion of my customers are asking for chicken fingers, which i don’t currently serve, but i’d like to serve that demand. my first instinct is not to take my existing burgers and to just turn them all into burgers that feature a beef patty + chicken fingers between the bun at 1.5x the cost and price of the original burger. instead, my first instinct would be to offer the burgers and chicken fingers as separate menu items, and let my customers order burgers and/or chicken fingers as they like.
and so i would apply the same thought process to this kind of system change. unless i’m absolutely certain the existing map styles don’t properly serve any audience or that it’s no longer reasonable to provide them for technical reasons, it seems like the wrong approach to take something that works for a lot of people and replace it with something significantly different just because there are a few folks asking for something different.
i’ll assume this is your interpretation of:
i think a better interpretation of technical complexity is not “too hard” but “benefit < cost”. i think a better interpretation of visual complexity is not “ugly” but “too many variables / parts” (kind of like how some people might like burgers and/or chicken fingers, but they all might not necessarily want to always order a combined beef patty + chicken finger burger).
In this metaphor a customer wants chicken fingers (ability to see map without being obstructed with wrong data), iNat is selling them a burger (obscured observation with fake location) and telling the customer it’s chicken fingers (zoomed out they are the same squares) and also making it really hard to actually get a burger - long lines on purpose = super cluttered maps. Also when asked every customer except you who bothered to respond said they wanted chicken fingers rather than a burger or else some hybrid feature (a chicken sandwich maybe). Perhaps the store still can’t sell chicken fingers for whatever reason but they shouldn’t be given a burger and told it’s chicken fingers nor is it correct to imply most customers want burgers based on only one person (you) wanting burgers. Perhaps the vast majority who didn’t respond at all do indeed want burgers but you can’t imply that from this thread, where 17 customers voted for chicken fingers and one guy in the corner is shouting that hamburgers are better and all of them actually want hamburgers. That guy doesn’t own the store and has no idea whether chicken fingers are viable to sell here.
This is somewhat related. I like to use the explore feature of INat to find new locations to visit. I found the obscured locations were cluttering the maps so I created a query that does not display obscured observations and I saved the query in my profile so that I have easy access to it.
Maybe. Honestly I don’t really have a problem with the way the obscured data is represented other than the fact that it’s very cluttered. I know that’s more upsetting to some than it is to me so I don’t really have strong feelings about it. I think it’s important that it’s represented somehow, so I wouldn’t be in favor of no representation but I don’t really think anybody would be. I hope there’s a satisfying solution. I definitely don’t have any possible suggestions though.
I would really really like to be able to easily toggle on/off obscured observations in the map view - I often use the maps to find areas near me that have few observations, and go there to try to document underdocumented areas. I’ve finally figured out how to turn them off using url wizardry, but it used to make it pretty much impossible.
