Skip to main content

Prevent cheating

Best practices against cheating​

While severely sanctioned by the school, cheating is still a problem. In this section, we will specifically discuss how to prevent exfiltration of testsuite data via exploiting the trace system.

How to attack a testsuite​

Since MaaS workers do not have access to the internet (and logs are not available for students), the main point of attack students have is the output provided by the trace.

Students can extract information such as:

  • Test cases
  • Test environment information
  • Reference implementation (if included in the testsuites)

All of this could be logged in one way or another to the output file.

So, how do we defend ourselves against all of this?

Splitting execution​

While this is not necessary possible for unit tests, functional tests can perform their operations in two steps:

  • A first stage where you execute the student's code and store the output in a file.
  • A second stage where you compare the output with what you expected and write this out as a trace

This avoids students rewriting their trace to whatever they want since their code will only execute in the first stage (which does not have access to the final trace file).

Avoid letting students manipulate their trace file​

Always avoid letting students write whatever they want to the resulting trace file. See above for how to do that with functional tests. For unit tests, check that your unit testing framework overwrites whatever file it is supposed to output to at the end of the testing session.

Restrict file I/O​

In order to avoid manipulation of the trace file, run the testsuite as a different user (using su) who has no right over the file system.

Frameworks leaking tests​

Some test frameworks (like Jest) output parts of the test file when a test fail. In order to avoid leaking details about the test cases that are checked, make sure you disable or circumvent that behavior.