For Teachers & Admins

Vocareum Notebook includes a built-in nbgrader-based autograder that grades student work using cell tags and solution markers directly inside the notebook. In many cases that is all you need. When your grading requirements go beyond what cell-tag grading supports — for example, testing a separate Python file the student submitted, grading code written in another programming language, running a subprocess, or incorporating AI-based feedback — a custom grading script gives you full control over the grading logic.

For setup instructions common to all lab types (file layout, grade and report files, Configure Workspace workflow), see Using Custom Grading Scripts in Vocareum Labs. Refer to Custom Grading Scripts Examples for sample code.

When to use a custom grading script

The built-in nbgrader-based autograder is the right choice for straightforward notebook assignments where all graded work lives in tagged cells. A custom grading script is the better fit when any of the following apply.

Scenario	Why a custom script helps
Students submit `.py`, `.r`, `.java` or other non-notebook files	The built-in autograder only operates on cell content; a script can import and test external files
You need subprocess-based testing	Scripts can invoke any shell command, test runner, or compiler
You want AI-powered grading or rubric feedback	Scripts can call the Vocareum GenAI Gateway to evaluate open-ended responses
You need to evaluate multiple files or check file structure	Scripts can inspect the entire submission directory
You want to combine built-in autograder scores with other criteria	A script can invoke the built-in grading pipeline and then add additional scoring

How custom scripts interact with the built-in autograder

Adding a custom grade.sh to an assignment part replaces the built-in nbgrader-based autograder for that part entirely. Vocareum detects the presence of the script and delegates all grading to it. The built-in cell-tag grading pipeline does not run.

If you want to preserve cell-based grading alongside your own logic, your custom script must invoke the grading pipeline explicitly. The section below covers this hybrid pattern.

Accessing the submitted notebook

In the grading environment, the student's submitted notebook is available in /voc/work/ (also accessible as $VOC_HOME_DIR):

$VOC_HOME_DIR/<notebook-name>.ipynb

The notebook name matches what the student sees in their workspace. You can reference it by name in your script, or discover it dynamically:

NOTEBOOK=$(find "${VOC_HOME_DIR:-/voc/work}" -maxdepth 1 -name "*.ipynb" | head -n 1)

To parse the notebook in Python, use the nbformat library:

import nbformat import os  work_dir = os.environ.get("VOC_HOME_DIR", "/voc/work")  with open(os.path.join(work_dir, "assignment.ipynb"), "r") as f:     nb = nbformat.read(f, as_version=4)  for cell in nb.cells:     if cell.cell_type == "code":         print(cell.source)

nbformat is available in the grading environment. If you need additional packages, see Installing Packages in Vocareum Lab Environments.

Note: Vocareum Notebook only includes code cells in the submitted notebook. Markdown cells are not passed to the grading environment. If your assignment requires students to provide written input — such as a short-answer response — have them assign it to a variable in a code cell rather than writing it in a markdown cell.

Executing the notebook programmatically

If your grading logic depends on the notebook being executed — for example, to capture cell outputs or to run the notebook's embedded tests — you can use nbconvert to execute it:

jupyter nbconvert --to notebook --execute \     --ExecutePreprocessor.timeout=120 \     --output "${VOC_HOME_DIR:-/voc/work}/executed.ipynb" \     "${VOC_HOME_DIR:-/voc/work}/assignment.ipynb"

The --ExecutePreprocessor.timeout flag controls how long execution may run before the command times out. Set this to at least 60 seconds for notebooks with meaningful computation; increase it further for assignments that use AI calls or long-running processes.

Using the built-in autograder alongside a custom script

To combine cell-tag grading with your own logic, execute the notebook with nbconvert and then parse the executed notebook's cell outputs to extract scores. A worked example of this hybrid pattern is provided in Custom Grading Scripts Examples.

Using Custom Grading Scripts in Vocareum Notebook