Not all observations uploading

I just re-indexed the unindexed observations after your latest post. I also tried again to replicate the indexing issue by making test observations with nearly your same attributes (same fields, added to the same project, etc) and still cannot replicate it.

2 Likes

Ken-ichi, I am recording the non-indexers here so that I have a record of them, not as a dig at you guys! I am totally grateful for all the help!

I can’t fave (or unfave) because the obs don’t load :-( The key to open the box is inside the box ;-)

1 Like

Thank you so much, Patrick!! I have checked them and they’re all there - yay!

On the one - https://www.inaturalist.org/observations/37027370 - I had the sex annotation female. When I tried to remove it now, it won’t go and the little wheel just spins. I waited 2 minutes and then refreshed the ob. The annotation was still there. Tried deleting it again, and it did delete after a few seconds.

Could there be a conflict between the field value and the annotation? If one tries to delete the annotation while the field value is still there, might the field value be trying to re-annotate at the same time? I deleted the field value after getting the annotation to finally delete. This may be a different problem altogether and not related to the non-indexing?

Sounds like a different problem. What we really need for the non-indexing problem is a list of steps to follow that consistently recreate the problem, e.g. “I upload this photo at this time with these values and every time I end up with an unindexed observation”

1 Like

I hear what you’re asking, but the problem is not consistent. Anyway, this is how it goes:

  1. Click on Upload
  2. Click on Choose Files
  3. Use shift-click (and sometimes ctrl-click) to select a bunch of files.
  4. Select cards with shift-click or ctrl-click, and click on Combine, or click-&-drag cards together. I use the first option if I’m happy that the first photo remains the first photo on the ob, and I use the second option is I want a photo other than the first photo as the first photo on the ob.
  5. Click on Select All and click on Locality. Type in a few letters into the pinned location field and wait for dropdown. Dropdown drops and disappears. Type in letters again. Dropdown drops down without filtering for locality. Type in letters again. Dropdown finds locality. Arrow down and dropdown disappears. Type in letters again. Dropdown finds locality. Click on it with mouse very quickly before it disappears again. If not fast enough - rinse and repeat. When I finally have the locality, click Update.
  6. I don’t have gps co-ords on pics, so I find them manually. I go back into the locality of each ob that is in a slightly different spot to the pinned locality, and move the pin.
  7. Click on Select All if all have the same habitat, otherwise shift-click or ctrl-click all obs with the same habitat. Then add Habitat (s Afr) field. Repeat for all different habitats in the batch.
  8. Click Select All, and add to Habitats (s Afr) project. Type H in the projects field, wait 15 seconds for the project name to appear in the dropdown, arrow down and it disappears. Type H again, wait for dropdown, click on Habitats project but by this stage all projects have suddenly dropped down and click on wrong project. Remove from that project. Type H in field again, and have mouse in position to click on the dropdown as soon as it appears. Success!
  9. Shift-click or Ctrl-click all insect adult obs and add field Insect Life Stage: Adult. Do the same for larvae, pupae, eggs and nymphs. Sometimes the fields disappear and I have to click out on the sidebar somewhere and then click back in the field to get them to appear again. This doesn’t always work, so I have to click on blank space, reselect the cards and try the field again. This is usually successful.
  10. Add obs to other projects if relevant, e.g. Nests & nesting (s Afr), Animated observations, Pillar Parade (s Afr) etc.
  11. Add taxon names to each observation, on its card. If more than one card has the same organism, then ctrl-click to select and add name in sidebar.
  12. Add text to descriptions where necessary.
  13. Add interaction fields if the other partner of the interaction is already uploaded and I have the URL. Add to Interactions (s Afr) project.
  14. Wait for all photos to upload.
  15. Click submit.
  16. Count the number of obs that have uploaded in the batch. If short, then refresh page (sometimes twice). If one or more obs are still missing, check the Calendar for that day. Open the missing obs. If they don’t load, then come here and add the URL as non-indexed for future reference.

All obs that have not indexed have been insects with Insect Life Stage: Adult field. I think this is where the problem lies. I will test tomorrow with batches without adding this field and see if the problem persists.

Thank you so much for your help!

1 Like

Have added 51 obs of insects this morning without adding life stage field, and lost none of them. Did all the other stuff as detailed above. So the non-indexing does seem to have something to do with the field and/or the annotation it populates?

It is also not messing up my order of obs, which was happening all the time before, and was a bit annoying because it all went out of order. Yes, yes, it is my mild ocd ;-D

1 Like

Have now added 126 obs of insects (a couple of spiders as well) without using the LSF, and all obs uploaded no problem.

1 Like

Ok, so just to be clear, we’re talking about https://www.inaturalist.org/observation_fields/325 with the value set to adult, right? As you’ve done at https://www.inaturalist.org/observations/37071606? I just tried uploading a photo with the Uploader and adding that observation field with that value and everything was fine, so while adding that field might be necessary to reproduce this bug, it doesn’t seem to be sufficient. Have you experimented with batches of different sizes? E.g. if you upload just one observation and add just that field, do you get the problem? Does it happen with every photo or just one? Does the batch need to include 10 photos? 20? Is there any correlation with time of day?

1 Like

i get the sense for some reason that the people who have experienced this problem are on the same kind of internet connection. i wonder if a slow / flaky connection can mess things up somehow? like if the base observation is loaded but then there’s a hiccup while waiting to create the associated child observation field record, then maybe that could cause this kind of issue? (just suspicions… haven’t actually dug through the code to see the sequence of what happens in the uploader.)

1 Like

Possibly related to this? https://forum.inaturalist.org/t/slow-loading-time-when-adding-annotations/8348/6

Ken-ichi, yes that field.

I have uploaded between 1 and perhaps 15 obs in a batch. I start losing an ob at 4 obs in a batch. Even if all obs in a batch are insects with LSF filled in, I only lose 1 ob (or 2 in two instances). If I upload a batch which includes plants, birds, mammals and insects, I’ve only ever lost an insect. I don’t add life stage or phenology fields to the other groups, only insects. I do sometimes add the Sex field to other groups.

Most obs that I upload have more than 1 photo. Initially, non-indexed obs had only 1 photo (as mentioned above), but on later losses they had more than 1. I can’t say exactly how many photos, but let’s say I upload 5 obs with 1 photo each, I still lose 1 ob.

On Brave I get no warnings of issues. On Firefox I did get warnings. I can’t use FF because it is desperately slow and basically doesn’t work on my bandwidth (either on Win or Ubuntu).

All non-indexed obs show in my Calender, but don’t open properly. The top menu of the page does load, and if there are notifications, they show. The footer also shows. But the space in between never populates.

When I first noticed this problem, I re-uploaded the pics and had no problems. So it’s not an image issue. I also emailed a photo to Tony, but he never mentioned there being a problem with it.

Time of day and day of the week make no difference.

On 4 January I uploaded a total of 152 obs - all insects (and a coupla spiders) without adding Life Stage or Sex fields, and lost none of them. Some obs had up to 20 photos (because cuckoo wasps are so gorgeous), and some had only 1 photo. When I’d submitted, they all showed instantly in my Edit Observations page without me having to refresh.

Hope this helps.

My non-indexing problem may be an extreme case of
https://forum.inaturalist.org/t/observations-failed-to-add-to-habitats-s-afr-project-in-web-uploader/8682
and
https://forum.inaturalist.org/t/observation-field-entries-rejected-during-obs-upload/9554

which all seem to have problems with Fields.

1 Like

https://mybroadband.co.za/news/internet/335662-second-cable-break-found-slow-internet-speeds-set-to-continue-in-south-africa.html

Slow internet till February.

I’m in an excellent speedy internet connection area right now, and Field info included with the observation at the time of upload is not being included. I have to go back in to the obs and add the field info again.

I’m uploading obs individually on the website (laptop), at any time of day/night (speedy internet) as I always have, and any/every Field info I fill in at the time of adding images, location, description etc is not being included during the upload. I have to go back into the observation and add the Field info again.

Tried uploading using the green UPLOAD button, top right of screen, or the blue Add Observations button top left of page on Your Observations page. Same problem. Photos before and after attached. You can see on the before image I’ve clearly filled in the Fields info, but once the obs is uploaded, no Fields info included. Sorry, after image is not sharp, but you can see no Fields are included.


1 Like

@jacqui-nz, that looks like a different issue, so please open a new thread for that. This thread is for observations not getting indexed (i.e. available in observation searches) after upload, not for observation fields not getting added.

Thanks. The new thread was already started, I made the comment on this topic in case the Observation Fields problems were related. https://forum.inaturalist.org/t/observation-field-entries-rejected-during-obs-upload/9554

1 Like

This might be another instance of the same problem:
https://forum.inaturalist.org/t/observations-failed-to-add-to-habitats-s-afr-project-in-web-uploader/8682/25

This bug seems to have been fixed. I can add Life Stage annotations during upload and I don’t have any indexing problems. I have tried all sorts of permutations, and iNat is no longer foxed by any of them ;-)

Thank you to the fixer(s)!

Will mark this as solved.

2 Likes