Grading with Jupyter notebook

For auto-grading Jupyter notebook you can always extract the source code using nbextract and then use the Vocareum grading APIs. 

For Python notebooks there is another solution available on Vocareum platform - 

  1. Select "Scripts and Resources". Select View=>Jupyter. This will launch Jupyter notebook server from the "work" directory.
  2. When you create a cell, you have following options for the source cells - 
  • Read-only - This cell will not be editable by the student. Before auto-grading this cell will be replaced by the original cell in case student submission has any changes to this cell.
  • Auto grader tests - This cell is identified as a test block. The student will not be able to edit the cell. When you select it, you will be asked to select a criterion that was specified in the Part setting section. When auto grading is executed, if an assertion is triggered in this cell, the student will be assigned maximum points for this rubric, 0 otherwise.

3. After you have created the notebook, switch to "Standard View" - 

5. Run the following command in the work directory - 

vocReleaseNotebook <notebook name>.ipynb

This command will process the notebook and create a version for the student and copy the student version to the starter code.

Special tags in starter code

Any section of python cell starting with ### BEGIN SOLUTION will be stripped out until ### END SOLUTION is encountered and will be replaced with # YOUR SOLUTION HERE

In the auto-grader test cell of the starter code, if the platform encounters ### BEGIN HIDDEN TESTS, the code following it will be removed until ### END HIDDEN TESTS is reached. Note that the removed section will be replaced before the auto-grader is run.

Auto-grading

There are two utilities functions that you need to add to your submission/grading scripts for auto-grading to run - 

  • vocCheckNotebook  <notebook>
    • This prepares the student notebook for grading. It checks if any of the auto-grade tests blocks have been modified and replaces them with the original block if needed.
  • vocGradeNotebook <notebook>
    • This script grades the notebook. 

Generating Feedback

When auto-grading is run a file <notebook name>_feedback.html is generated. You can call the following function to make this file as the report file which the student sees when they select Details=>View Grading Report.

vocReportFile <notebook name>_feedback.html

This file includes HTML representation of the original notebook submitted by the student. In addition the assertion messages generated by the auto-grade test block are included. You can use this to provide additional automated feedback to students. Only the message associated with the first assertion that is triggered is reported.

For example, your auto-graded test might include the following statement - 

assert <condition>, "please check your work again"
print('Your solution passed the tests.')
If the tests fails then student would see the message "please check your work again" and "Your solution passed the tests" if the test succeeded.

Still need help? Contact Us Contact Us