CIS benchmark evidence

CIS Benchmarks  are consensus-developed secure-configuration recommendations. CIS describes Level 1 as a broadly usable baseline and Level 2 as defense in depth that can affect utility or performance. Apply either profile in a test environment first.

OpenFactory can apply selected controls and run a catalog of machine checks. A passing run is evidence about that artifact, benchmark version, profile, and point in time. It is not a CIS certification, a guarantee about deployment settings, or proof of compliance with another framework.

Current remediation boundary

The cis-benchmarks feature has one full, pinned remediation path:

The Ubuntu path deliberately disables or adapts selected upstream rules to fit the offline-first-boot and image environment. The generated manifest records provenance and selected settings. These adaptations make exact evidence and exceptions essential.

security-hardening and os.security.hardening_level are separate general-hardening controls. They do not map automatically to a complete CIS Level 1 or Level 2 profile.

Benchmark execution

Scenarios can select an exact catalog and level:

{
  "id": "ubuntu-cis-evidence",
  "name": "Ubuntu 24.04 CIS evidence",
  "enabled": true,
  "tests": ["boot", "login", "packages", "services"],
  "cis_benchmark": "CIS_Ubuntu_Linux_24.04_LTS_Benchmark_v1.0.0",
  "cis_level": "L1"
}

The runner also has catalogs for Debian 12 and Debian 13 and an internal GxP-oriented check set. A catalog name is not proof of official CIS content or currentness. Confirm the catalog’s provenance, license, version, and control definitions before using it in an audit.

Do not use an older OS benchmark as a conformance claim for a newer release. The resolver currently has approved same-family fallbacks for some newer Ubuntu and Debian targets so tests can run, but the output must be labeled as a compatibility assessment against the older catalog, not as conformance to a benchmark for the newer OS.

Reading a result

Retain at least:

• artifact and build IDs;
• image digest and source snapshot;
• base OS identity and version;
• benchmark/catalog ID, profile, and digest;
• applicable, passed, failed, skipped, and not-applicable controls;
• raw command output and timestamps;
• remediation provenance;
• documented exceptions with owner and approval; and
• the environment in which the checks ran.

A percentage alone hides applicability and severity. A run with zero failed applicable checks still says nothing about controls outside the catalog, deployment-time changes, credentials, networks, physical hardware, or later drift.

Safe workflow

1. Select a benchmark that exactly matches the target release.
2. Review the profile and every OpenFactory adaptation.
3. Build the artifact and wait for first-boot remediation readiness.
4. Run the complete applicable check set in a clean guest.
5. Investigate failures; do not silently convert them to not applicable.
6. Test application and operational behavior after hardening.
7. Have the accountable security owner approve exceptions.
8. Re-run after any package, recipe, or deployment change.

Use CIS’s own profile guidance  as the normative starting point and obtain the applicable benchmark through your authorized CIS distribution channel.