Starting with Jobs
MaaS
MaaS is a job scheduler, similar to Nomad or a GitLab CI. It is used for running remote jobs that need to run in specific Docker containers. MaaS is hosted by the Forge and is used as the execution engine for the jobs.
When a submission is made against a submission definition in the activity, a job is created. This job leads to the creation of a MaaS pipeline, which uses the workflow specifications provided in the activity.
Each submission corresponds to a specific action taken by the student, whether it be creating a tag or submitting their work. These submissions trigger the creation of jobs, which, in turn, represent events directly targeting a submission. In other words, jobs are specific occurrences tied to the execution of a submission. It's noteworthy that multiple jobs can be associated with a single submission, particularly when jobs are retried, with each new attempt giving rise to a fresh job intricately linked to the original submission.
What jobs can do
Jobs can run any code you wish to test. You have control over the testsuite and the docker images that get ran in Docker containers.
The docker containers can be ran in parallel or sequentially (or both!), but you will always have an "input" container (which receives the input tarball) and an "output" container (which outputs a trace file in XML).
Optionally, the input tarball can be concatenated with a testsuite tarball defined within the documents of an assignment. See more on inputs
The idea is as follows:
What it cannot do
Containers ran in MaaS do not have access to the internet!
Containers cannot use arbitrary images. Images must be hosted in a specific Docker registry. This is defined in the MaaS tenant, which you do not have control over. This is not the same tenancy as the one used for the Forge. Contact us if you need help about this.
As stated above, containers do not have access to the Internet. This is to avoid having students downloading random files, or worse, uploading arbitrary files to the Internet (code ran in MaaS is considered unsafe). This may seem handy, but adds restrictions in toolchains that require downloading things (like dependencies) from the internet. See here for details on how to handle this restriction.
Jobs themselves do not output grades. Instead, they generate traces, which serve as concise summaries of test outcomes (passed or failed). These traces provides constructive feedback to students. To convert these traces to grades, you will need to leverage another tool designed for this purpose.