Monthly challenge grader system #107
Merged
+789
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jun 13, 2024
qosf.org/challenge_grading/.DS_Store
Outdated
There was a problem hiding this comment.
These files should be added to .gitignore. Make sure to remove these files before adding to .gitignore though.
qosf.org/challenge_grading/challenges/release/july/.ipynb_checkpoints/task1-checkpoint.ipynb
Outdated
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello,
This grading system proposal implements nbgrader for jupyter notebooks allowing automated grading and export of csv with the results. This collection of scripts will ease the job for creating, distributing and collecting solved notebooks. A example notebook is included with a simple test, it's a trivial qiskit challenge. This could be fully automated in some point but that will depends on your available or desired execution environment but it is a good start to run locally; for example, it could work with a CI triggered by pull requests in a github repo, upload of submissions in a website, or even a jupyter server as the now extint Qiskit Labs. The requirements for the jupyter notebooks with nbgrader metadata are simple, but as far as I know is not possible to create all-cases generic tests, the checks or assertions must be created carefuly to avoid the participants to reverse enginner the notebook or fake a correct answer and therefore the tests should be a combination of hidden and visible assertions to verify the correctness of the achievement.
There is a README.md for installing and usage.
Closes #94
[Unitary Hack 2024]