Release any orphaned claimed executions as part of the process prune #244
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Orphaned executions shouldn't happen as part of regular operation but some people have reported them. The only way I can think of this happening could be that a worker gets deregistered while still working normally, so that it happens to be claiming executions while the process record is deleted. This means that the callback to release claimed executions might not see yet the executions that are being claimed at the same time, and as such, it won't release any. Or, in other words, those executions would get claimed by a process that's getting deleted and that once they're committed to the DB, it no longer exists. To recover from this scenario, as well as checking for processes with an expired heartbeat, the supervisor will also check for orphaned executions and release them.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Orphaned executions shouldn't happen as part of regular operation, but some people have reported them (#159). The only way I can think of this happening could be that a worker gets deregistered while still working normally so that it happens to be claiming executions while the process record is deleted. This means that the callback to release claimed executions might not yet see the executions that are being claimed at the same time, and as such, it won't release any. Or, in other words, those executions would get claimed by a process that's getting deleted and that once they're committed to the DB, it no longer exists.
To recover from this scenario, as well as checking for processes with an expired heartbeat, the supervisor will also check for orphaned executions and release them.