Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This implements a simple self-stabilizing task selection system:
The goal of the algorithm
task_selection
is to balance prompting vs voting, such that the number of active prompts stays constant.Further, we try to get tasks done as soon as possible, meaning that in the case a prompt is ready for ranking, we will preferentially (with probability
p
) rank first.If the algorithm decides to "answer" a prompt, a second function
next_answer_task
exists that, given a dictionary of prompt-ids and existing number of answers, decides which prompt to provide an answer to next.This also prefers close-to-done prompts so that we can start ranking quickly.
If the selected task doesn't exist (i.e. there are no prompts to answer) I would just re-sample which task to choose at the moment (though with the automatic balancing I'm not entirely sure whether this is even possible).