For Teachers and Admins
Students can submit their work in Vocareum Notebook and have it be manually graded, auto-graded, or a combination of both. In this article, we will go over how to set up these grading features.
Notebook Cell Tags
Vocareum Notebook natively supports grading a Jupyter Notebook (.ipynb) file. Custom cell tags are used to identify manual and auto-graded cells within your notebook.
Vocareum's auto-grading system is compatible with nbgrader, an open-source tool for grading Jupyter Notebooks. If you already have notebooks that use nbgrader, you can upload them to Vocareum Notebook. If you have not used this notebook grading tool before, this article will guide you through the process of setting up grading in your assignments.
The available cell tags are as follows:
Manually graded answer: This cell is used to generate manual grading criteria.
Auto-graded answer: This is the solution block used in auto-grading. In the student's notebook, the solution will be hidden.
Auto-graded tests
This is the test block used for auto-grading.
You can add one or more auto-graded test cells for each auto-graded answer cell.
Read-only
This cell will not be editable by the learner.
Useful for cells containing lesson instructions or other course content that the student is not expected to modify.
Manual Grading
There are two ways to add rubric criteria for manual grading: you can create them directly in the assignment part settings or generate them by using cell tags in your notebook.
Add Criteria in Part Settings
From the Part settings of your Assignment, navigate to 'Rubrics' and select '+ Grading Criterion'.
Add a name to your criterion and set a Max Score.
Select 'Save Part' when you are done.
Now, when you visit a student submission, you will see the rubric options to grade. You can also leave a review comment, which the student will see once their submission has been graded.
Use Manually Graded Answer Cells
Select Configure Workspace from the assignment page to open Vocareum Notebook.
Right-click in the work folder to upload or create a new notebook.
Create a cell in your notebook and set the grading tag to "Manually graded answer".
Once selected, you will need to enter a name for the grading rubric item (also referred to as a criterion) and assign a point value. Note that the name and point value will be shown to the teacher when manually grading a student submission, and to the student when they view their grade.
When you are done setting up your notebook cells, proceed to the instructions in the Publish Notebook section.
Auto-Grading
Select Configure Workspace from the assignment page to open Vocareum Notebook.
Right-click in the work folder to upload or create a new notebook
Within your notebook, identify the cells you will be using for auto-graded tests. Cells tagged as "Autograded answer" should always be followed by one or more cells tagged as "Autograded tests."
Prepare the "Autograded answer" cell.
This is the solution block used in auto-grading.
With this grading type selected, anything you include between the special markers
### BEGIN SOLUTIONand### END SOLUTIONwill be hidden from the student in their version of the notebook and replaced with the prompt### YOUR CODE HERE.The code the student writes in this cell will be run against the auto-graded tests when they submit their assignment.
Prepare the "Autograded tests" cell.
This is the test block used for auto-grading.
You can add one or more auto-graded test cells for each auto-graded answer cell. Test cells must immediately follow the answer cell they are intended to evaluate.
When you select this tag, you will be asked to specify a rubric name and assign a point value.
Code you include between the special markers
### BEGIN HIDDEN TESTSand### END HIDDEN TESTSwill be hidden from the student.The learner will not be able to edit this cell in their own notebook.
Before auto-grading occurs, this cell in the student's submitted notebook will be replaced by the original cell containing your tests. However, when the student reviews their submission, they will see the version with the tests hidden.
The learner will be assigned the maximum number of points if all tests pass and no errors are thrown. If any errors occur in the cell during the grading process, the student will be assigned zero points for the corresponding rubric item.
When you are done setting up your notebook cells, proceed to the instructions in the Publish Notebook section.
Publish Notebook
After you have defined the notebook cells, you click the "Publish Notebook" button at the top of your lab.
You will be presented with a dialog box (shown below). Please read carefully and select the options as follows:
The following actions will be performed:
Place a modified copy of this notebook into /voc/startercode with the code between "SOLUTION" and "HIDDEN TESTS" pragmas removed.
An original copy of this notebook will be viewable by graders.
Copy any selected data files in the work area into the startercode folder.
If autograded cells are present:
Create or update Vocareum rubrics based on the names and points of autograded cells.
Generate a grading script for the autograder, with any existing script being overridden.
Select "Start" to deploy the notebook. Students should now see the notebook and any data files you selected during the publishing process in their "work" folder when they begin the assignment.
Submitting Assignments
When students are ready to submit their work they can click the 'Submit' button in their Vocareum Notebook assignment.
Their notebook will then be passed through Vocareum's grading servers and graded based on the tests you previously created. The grading script will auto-grade the cells marked as "Autograded answer" and will leave any manual grading criteria unscored until they are reviewed by a teacher.
When the grading process has finished, the student's submission panel within Vocareum Notebook will be updated with their grades, showing them their score breakdown for each generated rubric.