Print

Automatic tests



Bodek can run a simple presubmission test to validate a Student's submission.
The test is run immediately on submission, if there is a file named test in directory ~<course>/course-admin/bodek/assignments/<assignment>/presubmission.
The file can be either an executable file, or a tar file containing an executable file named test.
The test takes a single parameter - the path to a submission archive file. It ignores its standard input. The standard output and error are included both in the PDF and in the mail message sent to the Student.
On each run, Bodek provides an empty directory and an environment variable, BODEK_RESULTS_DIR, that points to it.
The contents of any text file created in this directory will be included as a separate chapter in the PDF.
(One should avoid special characters that may result in the file being recognized as type application/octet-stream rather than type text/*.)

Notes:
  • Directory presubmission is created automatically on the first submission. It should not be created manually.
  • Use one of the Test user accounts to check the test. See question 14. How can I test my course? in the FAQ for teachers
  • The test runs as a special temporary user. Any files that it needs (other than the archive file) should be world readable.
  • The test runs in a special directory containing only the test file (or untarred test files). It should not write anywhere other than to the local directory and to BODEK_RESULTS_DIR.
  • The test runs on a system similar to the the CSE public workstations and servers.
  • The test should not be long. It will be forcefully terminated after 300 seconds.
  • One should try running the test as a regular user (not the course user) to check that it runs correctly.
    If the test is not valid (i.e. not a proper tar or executable), mail will be sent to the course account when a Student submits.
  • File types are checked using MIME magic. See the file manpage.
  • One can use the presub utility to simulate a student submission, including generating the PDF. However, the test will be run as the current user on the local machine.
  • After a student has submitted, the Teacher can regenerate the PDF (and re-run the tests) by selecting Regenerate submission PDF from the Edit drop-down menu in the Moodle assignment's View/grade all submissions page. Mail is not sent to the student.

Configuration directives:

The test takes configuration directives from file profile, in the same directory as the test file.
Currently only one directive is supported: 
use_postgres = yes
This directive ensures that Bodek creates a temporary, empty PostgreSQL database and account.
Bodek provides the database to the test through the following environment variables:
PGHOSTThe name of the PostgreSQL server
PGDATABASEThe name of (temporary) PostgreSQL database
PGUSERThe name of the (temporary) PostgreSQL user
Note: These are the environment variables used by most PostgreSQL clients and client libraries,
Hence the utility psql can be used without any parameters. (It will, however, write an error regarding an uninitialized value $gid, which can safely be ignored.)