Please fill out the following sections to the best of your ability, it will help us investigate bugs if we have this information at the outset. Screenshots are especially helpful, so please provide those if you can.
Platform (Android, iOS, Website):
App version number, if a mobile app issue (shown under Settings or About):
Browser, if a website issue (Firefox, Chrome, etc) :
URLs (aka web addresses) of any relevant observations or pages:
Given that a dialog pops up when you click on the 334 page reading
Too Many Results
Page number times the number of results per page cannot exceed 10,000. Try applying filters to reduce the number of results, or mark observations as reviewed and use the “View More” button instead of pagination
I suspect it is an intentional, unannounced change, not a bug.
I think this was just implemented though, prior to the past couple of days you could see the full result set. Know this for sure as I was sadly monitoring the needs ID count for Canada going over the 50,000 pages total…
I was using that visible number to work thru 3 to 5 pages each day (depending on whether there was a page of sealife I could mark as reviewed, Next) Now I am a hamster spinning on an endless wheel … until I recognise the older obs, and know I have caught up with myself.
this is right. they added a new parameter on some of the API endpoints (&skip_total_hits=true) that allows you to limit the total record count to 10000. so i guess the 334 is a reflection of the limited count. not sure why this was even necessary, since they’re still effectively doing the counts to provide the metrics noted above.
While I dont work for the site, so you can combine my feedback with 2 bucks and go get yourself a coffee, I’m pretty sure the answer is going to be that it is not server capacity sustainable to keep serving up hundreds of thousands, or even millions of results every time someone clicks on the Identify page.
when they implemented the new feature that allows the total count to be cut off at 10000 (as part of performance improvements, it seems), they probably just didn’t anticipate that it would have this particular effect.
as mentioned, it seems like they’re effectively doing the count anyway to get the observation count metrics. so it seems like they could still calculate a dummy total page count, if it’s really going to bother folks.
that said, right now, they’re doing those counts regardless of whether or not the section is actually expanded / displayed, but maybe they intend to refrain from doing the counts if the section is hidden. in that case, it becomes more of a question of whether eliminating all the counts is more beneficial from a technical perspective or displaying a max page number is more beneficial. i’ll let the staff make that call.
I also used this “feature” to calculate how many observations of a given taxon were in needs-ID (or other criteria), so I could monitor ID progress over time. Even though it never allowed you to actually fetch over 10k records, it still showed the total number, so I could easily calculate e.g. total observations versus needs-ID for a given taxon. But the method provided by @pisum (thanks! I had forgotten that was even there) seems to work well enough for my purposes:
This was a change I made last week while trying to find ways to improve performance and reduce load on databases. Fetching counts across large results sets is rather costly (in terms of database resources and response time), so I was looking for places we could eliminate counts. Previously the Identify page would reference all pages in the links at the bottom, even though clicking on anything higher than 333 would result in an error (this is still true when you click on page 334). Given the links were guaranteed to give an error, and that we had total counts in the stats panel on the right, I made a judgement call to allow pages to be limited to 334, but clearly people were using those page counts for reasons other than pagination.
I can restore the page links, though they will still generate errors. I think ideally we can come up with a design that allows users to get the information they need while not generating links to things we know will generate errors. It also seems like at least the folks that have comments here for the most part don’t use or weren’t aware of the slideout stats panel that lists the total number of observations and number of reviewed observations in the current query. If that panel is not widely used, we could save an additional query by not counting the number of observations the current user has reviewed. I won’t make that change, but I am curious to know if people are using it, and if so is the number of reviewed observations helpful.
It sounds like several people want to know the total number of observations in a search and they were using the number of pages to approximate that. If we changed the interface to for example say there are 333 pages accessible, but also included the total count more prominently, would that meet your needs? That way we could at least avoid these page links we know will generate errors.
Are exact counts very important, or could there be some cap, or some rounding. For example if there are more than 100k results, could it say “more than 100k results”? Or if there are 123k results, could it say “there are about 120k results”?
Lastly - is anyone using the total count of reviewed observations we show in the side panel, or is something we could phase out? Sounds like several people didn’t use that panel or perhaps knew it was there.
For now I’ll restore the page links. Thanks and apologies for the unexpected change.
If I use the slideout panel - only to see my progress against the running total - while it is open, that tiny bit of information carves a whole column of obs off that side of my screen. A tiny mouseover popup would be adequate for me. And that popup could live at the bottom with the existing page numbers. No slideout needed.
At least, please not stuck at a confusing 334. That isn’t information we can use. The (but wait there’s more) arrow is fine.
I used to use it, but I forgot it even existed within months of its being hidden.
Yes, and I find that helpful. However, what bothers me much more is that the popup upon reaching the end of a page now also says 10,000 when I know there should be more. I just want a quick and easy way to see what the total number of results is; I don’t really care how or where.
When the stats panel / sidebar also contained the leaderboard I found it more useful, but when the leaderboards were phased out I just permanently collapsed that panel and forgot about it since it was basically empty after that change and takes up space. Personally I have never used the “# reviewed” section in that panel (since I can extract that # in other ways) but that’s just me and I imagine everyone finds their own way to get the info/workflow they need. From a UI/UX perspective, more available options is often better since it allows everyone to find what feels comfortable for them.
I have definitely used the exact numbers when trying to calculate % of IDed observations in a given taxon (for example)… Rounding a large number like 768,801 to the closest thousand (or whatever) would be fine, but rounding 1501 to 2000 would probably impact functionality for some people. But it wouldn’t seriously break anything for me - having a responsive interface is overall more important to me than small issues like this. I will say it would be nice to have somewhere to get these numbers, even if it’s not on the default identify modal - maybe a checkbox for “show exact numbers” or something. That might reduce the overall DB load but still keep the data available for the probably-small % of people who want it there.