Limiting and scheduling large taxonomy ancestry changes

Patrick fixed that bug last week where in some cases only observations on the taxon we’re getting reindexed and not observations on descendants after an ancestry change.

But we realized that that bug was actually shielding us from an even larger reindexing load from these ancestry changes which is part of what prompted the limits describe here.

As for existing corruption in the index resulting from this now fixed bug, if you see branches with the index being out of sync (eg ants) let us know and we’ll reindex that branch.

We used to occasionally reindex the whole tree to clean up any drift that occurred as it took just a few hours. now it takes weeks so it’s not really possible given the slow site side effects. We’re working on ways to try to speed that up, but as the site gets bigger the problem gets harder so we think these limits are a pod idea regardless.

4 Likes