Assertion Types

Complete reference for all test assertion types.

Overview

Assertions verify specific conditions in your built image. Each assertion has a type and parameters.

{
  "type": "assertion_type",
  "params": {
    "key": "value"
  }
}

User Assertions

user_exists

Verify a user account exists.

{
  "type": "user_exists",
  "params": {
    "username": "deploy"
  }
}

user_in_group

Verify a user is a member of a group.

{
  "type": "user_in_group",
  "params": {
    "username": "deploy",
    "group": "docker"
  }
}

user_password

Verify a user has a password set (a valid password hash exists, not a locked or empty account).

{
  "type": "user_password",
  "params": {
    "username": "deploy"
  }
}

Service Assertions

service_running

Check a systemd service is active.

{
  "type": "service_running",
  "params": {
    "service": "nginx"
  }
}

service_enabled

Check a service is enabled at boot.

{
  "type": "service_enabled",
  "params": {
    "service": "docker"
  }
}

To verify a running process rather than a service unit, use gui_application_process (see below) or command_succeeds with a pgrep command.

Package Assertions

package_installed

Verify a package is installed.

{
  "type": "package_installed",
  "params": {
    "package": "nginx"
  }
}

Works on both APT-based (dpkg) and RPM-based images. To verify a package is absent, use command_succeeds with a negated check, for example ! dpkg -l telnet 2>/dev/null | grep -q '^ii'.

Network Assertions

port_listening

Check a port is open.

{
  "type": "port_listening",
  "params": {
    "port": 443,
    "protocol": "tcp"
  }
}

network_reachable

Verify a host responds to ping.

{
  "type": "network_reachable",
  "params": {
    "host": "10.0.0.1",
    "count": 3
  }
}

In multi-VM scenarios, host can be another VM’s name; it is resolved to that VM’s address at run time.

http_responds

Verify an HTTP endpoint responds.

{
  "type": "http_responds",
  "params": {
    "url": "http://localhost:8080/health",
    "status": 200
  }
}

In multi-VM scenarios, VM names inside the URL (for example http://node2:8080/) are resolved to that VM’s address at run time. To assert on the response body, use command_output with a curl command.

File Assertions

file_exists

Check a file or directory exists.

{
  "type": "file_exists",
  "params": {
    "path": "/etc/myapp/config.yml"
  }
}

To verify a file is absent, use command_succeeds with test ! -e /path.

file_permissions

Check a file’s permission mode.

{
  "type": "file_permissions",
  "params": {
    "path": "/etc/ssh/sshd_config",
    "mode": "0600"
  }
}

file_contains

Check file contents.

{
  "type": "file_contains",
  "params": {
    "path": "/etc/ssh/sshd_config",
    "content": "PermitRootLogin no"
  }
}

Command Assertions

command_output

Run a command and check output.

{
  "type": "command_output",
  "params": {
    "command": "docker --version",
    "contains": "Docker version"
  }
}

command_succeeds

Verify command exits successfully.

{
  "type": "command_succeeds",
  "params": {
    "command": "systemctl is-active nginx"
  }
}

GUI & Display Testing

gui_application_opens

Launch a GUI application and verify its window appears.

{
  "type": "gui_application_opens",
  "params": {
    "command": "firefox",
    "window_title": "Firefox",
    "timeout": 10
  }
}

Use case: Verify EDC clients, imaging viewers, or CTMS interfaces launch successfully.

gui_window_visible

Check if a window with specific title is visible on screen.

{
  "type": "gui_window_visible",
  "params": {
    "window_title": "EDC Login"
  }
}

Use case: Verify login screens, confirm dialogs, or error modals display.

gui_execute_command

Execute a command in the terminal and capture a VNC screenshot.

{
  "type": "gui_execute_command",
  "params": {
    "command": "weasis /opt/test-data/chest-xray.dcm",
    "wait_seconds": 5
  }
}

Use case: Visual regression testing, UI state validation. Automatically captures screenshot + metadata.

gui_application_process

Verify an application process is running.

{
  "type": "gui_application_process",
  "params": {
    "process_name": "firefox"
  }
}

Use case: Ensure EDC client remains active, detect application crashes.

gui_screenshot_matches

Compare current screenshot against a reference image for visual regression testing.

{
  "type": "gui_screenshot_matches",
  "params": {
    "reference_id": "ecrf_demographics_page",
    "threshold": 0.95,
    "ignore_regions": [[10, 10, 200, 30]],
    "match_area": [0, 0, 1920, 1080],
    "save_as_reference": false,
    "wait_before": 1.0
  }
}

Use case: Detect UI regressions in EDC forms, verify DICOM viewer displays correctly, validate report layouts.

Storage: Reference images are kept per reference_id with metadata, diff images, and comparison history alongside your test results.

gui_click_element

Click a UI element at specified screen coordinates.

{
  "type": "gui_click_element",
  "params": {
    "x": 600,
    "y": 400,
    "button": "left",
    "double_click": false,
    "wait_after": 0.5,
    "verify_click": true
  }
}

Use case: Navigate EDC forms, click dialog buttons, interact with custom UI controls.

gui_form_fill

Fill form fields and optionally submit.

{
  "type": "gui_form_fill",
  "params": {
    "fields": [
      {
        "name": "patient_id",
        "value": "PT-001",
        "method": "tab"
      },
      {
        "name": "visit_date",
        "value": "2024-01-15",
        "method": "click",
        "x": 300,
        "y": 250,
        "wait_after": 0.3
      }
    ],
    "submit_button": {"x": 600, "y": 500},
    "wait_after_submit": 1.0,
    "verify_submission": true
  }
}

Field object:

• name - Field identifier (for logging)
• value - Text to enter
• method - tab (press Tab) or click (click coordinates)
• x, y - Coordinates if method=click
• wait_after - Seconds to wait after field (default: 0.3)

Use case: Automated data entry for eCRF forms, patient enrollment workflows, adverse event reporting.

gui_text_visible

Check if text is visible on screen using OCR.

Requires: pytesseract package and tesseract-ocr system package.

{
  "type": "gui_text_visible",
  "params": {
    "text": "Consent Form Approved",
    "region": [100, 100, 800, 600],
    "case_sensitive": false,
    "lang": "eng",
    "wait_before": 1.0
  }
}

Use case: Verify confirmation messages, validate report headers, check status indicators in legacy applications.

Installation:

pip install pytesseract
sudo apt-get install tesseract-ocr  # Ubuntu/Debian
sudo dnf install tesseract           # Fedora

Security Assertions

cis_benchmark

Run CIS benchmark checks.

{
  "type": "cis_benchmark",
  "params": {
    "level": 1,
    "profile": "server",
    "controls": ["1.1.1.1", "5.2.1"]
  }
}

The benchmark document is selected automatically from the image’s base (for example, CIS Debian Linux 12 for Debian 12 images). See CIS Benchmarks.

To verify individual firewall rules, use command_output against ufw status or nft list ruleset.