Edit project breaks entirely for project with undefined user

i’m not sure if this is what’s going on here, but one thing i noticed when making requests to other endpoints, i get errors if my payload contains extra trailing commas in the structure.

so something like this might not work:

{
  "record": {
    "item": "blah",
    "type": "huh",
  },
}

… but this would (or something like that):

{
  "record": {
    "item": "blah",
    "type": "huh"
  }
}

i didn’t actually verify this just now, but it was just an odd thing that i remember seeing, and your payload seems to have the same kind of potential issue maybe… i don’t have time to get into it right now, but i might be able to try a few things later.

Because the trailing commas is just a style I use in constructing the python dict, I don’t think it makes a difference to the end result. i.e. with / without them, requests is still going to see the same dict …

Something I was investigating in Discord chat with @jcook was whether this endpoint strictly requires Content-Type: multipart/form-data. If it does, then I have a bit of a tricky encoding issue, because I think I need to flatten the nested dict/array/dict structure into something that can then be encoded like that. Everyone seems to have their own solutions (various things found on stackoverflow) and none of them seem to do the job for me. Or I could just be entirely wrong about that. I might just want to quit now and see if Jordan can dig up something for me later when he has time.

Alas, I haven’t gotten any farther than you have, aside from discovering all kinds of ways to not send these requests. I’ll make some more attempts tomorrow.

1 Like

this appears to be possible. i was able to change a test project by including only the items that i wanted to change in the payload:

  1. change the project description (to “test”):
    curl "https://api.inaturalist.org/v1/projects/pisum-s-personal-project" -X "PUT" -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: JWT" -d "{\"project\":{\"description\":\"test\"}}"

  2. delete a particular user (1 of 2) from the “include” list (note destroy=true, and does not work if project_observation_rules_attributes[i].id is excluded):
    curl "https://api.inaturalist.org/v1/projects/pisum-s-personal-project" -X "PUT" -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: JWT" -d "{\"project\":{\"project_observation_rules_attributes\":[{\"operator\":\"observed_by_user?\",\"operand_type\":\"User\",\"operand_id\":779571,\"id\":831929,\"_destroy\":true}]}}"

  3. add the user back to the “include” list:
    curl "https://api.inaturalist.org/v1/projects/pisum-s-personal-project" -X "PUT" -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: JWT" -d "{\"project\":{\"project_observation_rules_attributes\":[{\"operator\":\"observed_by_user?\",\"operand_type\":\"User\",\"operand_id\":779571}]}}"

hope this helps. let me know if you need anything else.

2 Likes

Argh. OK, solved [edit: to clarify, I mean this subproblem is solved … the underlying bug isn’t]. I had not obtained the JWT with /users/api_token from the password token. This post sorted it out for me: https://forum.inaturalist.org/t/how-does-one-turn-an-authorization-code-into-a-jwt/13528

And then my test worked in curl. I’m reworking my python code snippet to do it that way.

1 Like

here’s just a little more related info:

2 Likes

Ah, good. So by always passing a JWT, everything will just work.

I’m back to not being able to fix 2020 because of a bit of key-in-locked-box problem. If I could edit the project with dronefly account, I would, but dronefly isn’t an admin for the 2020 yearlisting project, being one of our older projects, so it can’t edit it - but until I can run some code from an admin account, I can’t add more admins!

I have applied to make a new application under my own account, which is an admin of that project, so I can finally fix it, but now I have to wait for that application to be manually reviewed and approved. Sigh.

Anyway, eventually I’ll get there and get the project fixed. :) Thanks for all your help!

1 Like

i think that approval process should be quick (or at least it was for me). if you just need a JWT to run a quick script to update your admins using your own account in the meantime, you can always just go to https://www.inaturalist.org/users/api_token in your browser and manually get one there. (or you could repurpose one generated by other means.)

1 Like

Of course! Thanks. The deleted user is now gone from the project, and edit works again.

1 Like

great. i’m glad your specific project is now unstuck. i guess the general issue of deleted users in the “include” list of collection projects preventing the project rule edit page from loading properly is still a thing though. so bug still unresolved, in case it’s unclear for others looking at the thread.

I tried to replicate this issue but couldn’t. Here are the steps I followed:

  1. With tiwane, I made a collection project and included one of my test accounts (stafftest2, since deleted) as a User requirement.
  2. With stafftest2, I joined the project and I also uploaded an observation that met the project’s requirements.
  3. Checked that stafftest2’s obs was in the project and that stafftest2 was a project member.
  4. Deleted stafftest2 account.
  5. With tiwane, edited project and the edit page loaded fine.

Am I missing something?

i think stafftest2 needed to be included in the “include” user list, too. i only see tiwane and inatstafftestaccount in the list:

and just to verify, i saw only tiwane and inatstafftestaccount in the project rules of the API response (https://api.inaturalist.org/v1/projects/test-deleted-user-project?rule_details=true), too. here’s a snippet:

"project_observation_rules": [
  {
    "operand_id": 28,
    "id": 837292,
    "operand_type": "User",
    "operator": "observed_by_user?",
    "user": {
      "id": 28,
      "login": "tiwane",
      "spam": false,
      "suspended": false,
      "created_at": "2008-04-03T02:33:23+00:00",
      "login_autocomplete": "tiwane",
      "login_exact": "tiwane",
      "name": "Tony Iwane",
      "name_autocomplete": "Tony Iwane",
      ...
    }
  },
  {
    "operand_id": 180992,
    "id": 837294,
    "operand_type": "User",
    "operator": "observed_by_user?",
    "user": {
      "id": 180992,
      "login": "inatstafftestaccount",
      "spam": false,
      "suspended": false,
      "created_at": "2016-02-18T18:49:28+00:00",
      "login_autocomplete": "inatstafftestaccount",
      "login_exact": "inatstafftestaccount",
      "name": "",
      "name_autocomplete": "",
      ...
    }
  },
  ...
}

It was on the list. It disappeared after I deleted the account.

hmmm… that’s interesting. are there different ways to delete a user? if so, can you tell how the problematic user (1142264) from benarmstrong’s case was deleted? (maybe some methods of user deletion clean up associated records better than others?)

1 Like

Or has the code that cleans up records changed over the time between our last project update and when I discovered the issue and finally fixed it by removing the bad record?

This user has not been an iNat Discord community member for quite a while, and we haven’t updated the 2020 yearlisting much (maybe not even at all, apart from my recent update to fix it) since the start of 2021. So that’s a sizeable timeframe in which the damage may have taken place.

i guess if user cleanup is more robust now, then that might indirectly have solved the problem here. but probably iNat staff will be the only ones who can efficiently find out if there are other projects with deleted users in their include lists, and that could shed light on whether the problem might still be occurring.

1 Like

Could it be that deleting a user when there is a rule to include their observations but they haven’t joined the project causes the problem?

I also tried that and couldn’t replicate.

just to make sure the problem was what we thought it was, here’s another project with the same user in the include user list: https://www.inaturalist.org/projects/discord-inaturalist-june-2020-weekendblitz. @benarmstrong – can you check out the project edit page and see if you run into the same error? (let’s keep the rules on this project as is for now so that tiwane can look at it, if needed.)

1 Like

Yes, it breaks exactly the same way when I click Edit.