Standard Code Test

Standard code tests are dialog driven, where you specify input data and the expected output. Codio then executes the student code, supplies the specified input data, and compares the expected output to the student code’s actual output.

Note

You can also write code tests that give you in-depth control by allowing you to write your own code to execute tests. See Advanced Code Tests for more information.

Codio provides a Starter Pack project that contains examples for all assessment types and a guides authoring cheat sheet. Go to Starter Packs and search for Demo Guides and Assessments if not already loaded in your My Projects area. Click Use Pack and then Create to install it to your Codio account.

For more information about adding a Standard Code Test, view this video

Follow these steps to set up a standard code test:

  1. On the General page, enter the following information:

    General
    • Name - Enter a short name that describes the test. This name is displayed in the teacher dashboard so the name should reflect the challenge and thereby be clear when reviewing.

      Toggle the Show Name setting to hide the name in the challenge text the student sees.

    • Instructions - Enter text that is shown to the student using optional Markdown formatting.

  2. Click Execution in the navigation pane and complete the following information:

    Execution

    Note

    If you store the assessment scripts in the .guides/secure folder, they run securely and students cannot see the script or the files in the folder. The files can be dragged and dropped from the File Tree into the field to automatically populate the necessary execution and run code.

  • Timeout - Where you can amend the timeout setting for the code to execute. Arrows will allow you to set max 300 (sec), if you require longer, you can manually enter the timeout period.

  • Command - Enter the command that executes the student code. This is usually a run command.

  • Pre-exec command - Enter the command to execute before each test is run. This is usually a compile command.

    • Java

      Compile: javac -cp path/to/file filename.java

      Run: java -cp path/to/file filename

    • Python

      Run: python path/to/file/filename.py

    • C

      Compile: gcc filename.c -o filename -lm

      Run: ./filename

    • C++

      Compile: g++ -o filename filename.cpp

      Run: ./filename

    • Ruby

      Run: ruby filename.rb

    • Bash

      Run: bash full_path.sh

    • SQL

      Run: python .guides/scripts/helper_mssql.py –db_host localhost –db_user SA –db_pass YourPassword –db_name DBNAME

      Note

      First you must use Tools > Install Software to install the appropriate helper script for your database (MSSQL,MySql,PostgreSQL). For example, if you are using MSSQL, you would download the Helper MSSql.

      Install SQL Helper Script
  1. Click Grading in the left navigation pane and complete the following fields:

    Grading
  • Points - The score given to the student if the code test passes. You can enter any positive numeric value. If this assessment should not be graded, enter 0 points.

  • Allow Partial Points - Toggle to enable partial points, the grade is then based on the percentage of test cases the code passes. See Allow Partial Points for more information.

  • Case Insensitive - Toggle to enable a case insensitive output comparison. By default, the comparison is case sensitive.

  • Ignore White Space - Toggle to enable stripping out any white space characters (carriage return, line feed, tabs, etc.) from both the expected output and the student output.

  • Substring Match - Toggle to enable substring match when comparing the expected output to the student output. The entire expected ouput needs to be contiguous in the student output.

  • One Attempt Only - Toggle to enable the assessment to run only once. The student will be warned that they cannot resubmit the assessment. It’s recommended that you provide a Run Button for the student to test the code before running the actual assessment.

  • Add Item to Check - Click to create another set of input/output fields.

  • Input - Arguments - Enter the command line arguments that are read by the student code.

    Input Arguments
  • Input - Stdin - Enter data that would be entered manually in the console. For example, Enter your Name:. If using this input method:

    • The input data should have a new line if it would be expected in the actual program execution.

    • In the Output field, the prompt text that is displayed to the user appears in stdout and should be reflected in your output field but without the data entered by the user. You do not need a new line in the output field between each input prompt as the new line character is part of the user input.

    • Ignore white space and Substring match are recommended options as they make the test more tolerant. The following image shows how to format input and output fields if you are not ignoring white space or doing a Substring match. Note how the input field only supplied the values to be input, not the prompt itself (which is actually a part of stdout). It is important to accurately account for all spaces and carriage returns.

      Input and Output Example

    The following image shows the more tolerant approach with the Ignore whitespace option set. In this case everything on its own line for readability. The whitespace characters will be stripped out of both the expected output and the student output at runtime.

    Ignore Whitespace
  • Generate Item - Click this button and enter the following information to generate the item to be checked by your code:

    Generate Items
  • Show Error Feedback - Toggle to enable feedback about issues in the student’s code.

    Show Error Feedback
  • Show Expected Answer - Toggle to show the students the expected output when they have submitted an answer for the question. To suppress expected output, disable the setting.

  • Show Answer and Rationale to Students - Toggle to display the answer, and the rationale for the answer, to the student. This guidance information will be shown to students after they have submitted their answer and any time they view the assignment after marking it as completed. You can set when to show this selecting from Never, After x attempts, If score is greater than or equal to a % of the total or Always

  • Answer and Rationale - Enter guidance for the assessment. This is always visible to the teacher when the project is opened in the course or when opening the student’s project.

  1. Click Metadata in the left navigation pane and complete the following fields:

    Metadata
  • Bloom’s Level - Click the drop-down and choose the level of Bloom’s Taxonomy: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ for the current assessement.

  • Learning Objectives The objectives are the specific educational goal of the current assessment. Typically, objectives begin with Students Will Be Able To (SWBAT). For example, if an assessment asks the student to predict the output of a recursive code segment, then the Learning Objectives could be SWBAT follow the flow of recursive execution.

  • Tags - The Content and Programming Language tags are provided and required. To add another tag, click Add Tag and enter the name and values.

  1. Click Files in the left navigation pane and check the check boxes for additional external files to be included with the assessment. The files are then included in the Additional content list.

    Files
  2. Click Create to complete the process.