Default Tests

Testing is enabled by default in a normalized recipe, with four predefined test names: boot, login, packages, and network. These are small guest checks, not a complete acceptance suite. A recipe can disable testing or replace the list, and some artifact types use a narrower contract.

What the defaults actually prove

Test	Executed check	Does not prove
`boot`	`uptime` exits successfully through the QEMU guest agent	Every bootloader path, hardware boot, or absence of kernel warnings
`login`	`whoami` can run through the root guest-agent channel	Interactive password, SSH-key, PAM, desktop-greeter, or end-user login
`packages`	One of `dpkg --list`, `rpm -qa`, or `pacman -Q` succeeds	That every package requested by the prompt is installed
`network`	DNS resolves `example.com` and a TCP connection to port 443 succeeds after bounded retries	General internet health, HTTP content, every interface, or an air-gapped design

The Ubuntu 24.04 CIS path removes the generic network check and uses its readiness-gated feature assertion instead. A recipe without an expected internet route can have a failed generic network check converted to a warning.

Feature and capability checks

The runner also merges checks supplied by enabled feature metadata, legacy hook test files, the frozen capability plan, explicit custom tests, and any benchmark tests. Duplicate assertions are removed. Assertions with known missing parameters are dropped with a server warning; review the resulting test plan so a dropped check is not mistaken for a pass.

Examples include checking the configured SSH port, service state, a requested package, or a recipe-specific artifact. Coverage depends on the enabled feature: inclusion of a feature does not by itself guarantee that feature has a complete assertion set.

Reading the result

Use the assertion rows and evidence, not only the aggregate badge:

passed means the executable check returned its expected result.
failed means the check ran and contradicted the expectation.
error means the harness could not produce a verdict.
skipped means the check did not run, for example because its target VM was absent.

Errors and skips are not passes. A completed image and a passed test run are separate states; certification is a third gate.

Define stronger acceptance criteria

For a real workload, add prompt-relevant assertions. For example:


{
  "description": "Nginx serves the local health endpoint",
  "assertions": [
    {
      "type": "service_running",
      "description": "Nginx is active",
      "params": { "service": "nginx" }
    },
    {
      "type": "http_responds",
      "description": "Health endpoint returns 200",
      "params": { "url": "http://localhost/health", "status": 200 }
    }
  ]
}

Then inspect the built image in the console and, for hardware-bound claims, test on representative hardware. See Custom assertions and Assertion types.