Skip to Content
ReferenceAssertion Types

Assertion Types

Complete reference for all test assertion types.

Overview

Assertions verify specific conditions in your built image. Each assertion has a type and parameters.

{ "type": "assertion_type", "params": { "key": "value" } }

User Assertions

user_exists

Verify a user account exists.

{ "type": "user_exists", "params": { "username": "deploy", "groups": ["docker", "sudo"], "shell": "/bin/bash", "home": "/home/deploy" } }
ParameterTypeRequiredDescription
usernamestringYesUsername to check
groupsstring[]NoRequired group memberships
shellstringNoExpected login shell
homestringNoExpected home directory

group_exists

Verify a group exists.

{ "type": "group_exists", "params": { "name": "myapp", "gid": 1001 } }
ParameterTypeRequiredDescription
namestringYesGroup name
gidnumberNoExpected GID

Service Assertions

service_running

Check a systemd service is active.

{ "type": "service_running", "params": { "service": "nginx" } }
ParameterTypeRequiredDescription
servicestringYesService unit name

service_enabled

Check a service is enabled at boot.

{ "type": "service_enabled", "params": { "service": "docker" } }
ParameterTypeRequiredDescription
servicestringYesService unit name

process_running

Check a process is running.

{ "type": "process_running", "params": { "name": "nginx", "user": "www-data", "count": 4 } }
ParameterTypeRequiredDescription
namestringYesProcess name
userstringNoExpected owner
countnumberNoExpected process count

Package Assertions

package_installed

Verify a package is installed.

{ "type": "package_installed", "params": { "package": "nginx", "version": "1.24" } }
ParameterTypeRequiredDescription
packagestringYesPackage name
versionstringNoVersion prefix to match

package_not_installed

Verify a package is NOT installed.

{ "type": "package_not_installed", "params": { "package": "telnet" } }
ParameterTypeRequiredDescription
packagestringYesPackage name

Network Assertions

port_listening

Check a port is open.

{ "type": "port_listening", "params": { "port": 443, "protocol": "tcp", "address": "0.0.0.0" } }
ParameterTypeRequiredDescription
portnumberYesPort number
protocolstringNotcp or udp (default: tcp)
addressstringNoBind address

port_not_listening

Verify a port is NOT open.

{ "type": "port_not_listening", "params": { "port": 23 } }
ParameterTypeRequiredDescription
portnumberYesPort number
protocolstringNotcp or udp

http_responds

Verify an HTTP endpoint responds.

{ "type": "http_responds", "params": { "url": "http://localhost:8080/health", "status": 200, "contains": "healthy", "method": "GET", "timeout": 30, "headers": { "Authorization": "Bearer token" } } }
ParameterTypeRequiredDescription
urlstringYesURL to request
statusnumberNoExpected HTTP status (default: 200)
containsstringNoResponse body must contain
methodstringNoHTTP method (default: GET)
timeoutnumberNoRequest timeout in seconds
headersobjectNoRequest headers

dns_resolves

Verify DNS resolution works.

{ "type": "dns_resolves", "params": { "hostname": "example.com", "record_type": "A" } }
ParameterTypeRequiredDescription
hostnamestringYesHostname to resolve
record_typestringNoA, AAAA, MX, etc.

File Assertions

file_exists

Check a file exists.

{ "type": "file_exists", "params": { "path": "/etc/myapp/config.yml", "mode": "0644", "owner": "root", "group": "root" } }
ParameterTypeRequiredDescription
pathstringYesFile path
modestringNoExpected permissions
ownerstringNoExpected owner
groupstringNoExpected group

file_not_exists

Verify a file does NOT exist.

{ "type": "file_not_exists", "params": { "path": "/etc/passwd-" } }
ParameterTypeRequiredDescription
pathstringYesFile path

file_contains

Check file contents.

{ "type": "file_contains", "params": { "path": "/etc/ssh/sshd_config", "content": "PermitRootLogin no", "regex": false } }
ParameterTypeRequiredDescription
pathstringYesFile path
contentstringYesContent to find
regexbooleanNoTreat as regex

directory_exists

Check a directory exists.

{ "type": "directory_exists", "params": { "path": "/var/log/myapp", "mode": "0755" } }
ParameterTypeRequiredDescription
pathstringYesDirectory path
modestringNoExpected permissions

mount_exists

Verify a mount point exists.

{ "type": "mount_exists", "params": { "path": "/data", "fstype": "ext4", "options": ["noexec"] } }
ParameterTypeRequiredDescription
pathstringYesMount point
fstypestringNoFilesystem type
optionsstring[]NoMount options

Command Assertions

command_output

Run a command and check output.

{ "type": "command_output", "params": { "command": "docker --version", "contains": "Docker version", "exit_code": 0, "timeout": 30 } }
ParameterTypeRequiredDescription
commandstringYesCommand to run
containsstringNoOutput must contain
exit_codenumberNoExpected exit code
timeoutnumberNoTimeout in seconds
regexbooleanNoTreat contains as regex

command_succeeds

Verify command exits successfully.

{ "type": "command_succeeds", "params": { "command": "systemctl is-active nginx" } }
ParameterTypeRequiredDescription
commandstringYesCommand to run
timeoutnumberNoTimeout in seconds

GUI & Display Testing

gui_application_opens

Launch a GUI application and verify its window appears.

{ "type": "gui_application_opens", "params": { "command": "firefox", "window_title": "Firefox", "timeout": 10 } }
ParameterTypeRequiredDescription
commandstringYesCommand to launch application
window_titlestringYesExpected window title
timeoutnumberNoSeconds to wait for window (default: 10)

Use case: Verify EDC clients, imaging viewers, or CTMS interfaces launch successfully.

gui_window_visible

Check if a window with specific title is visible on screen.

{ "type": "gui_window_visible", "params": { "window_title": "EDC Login" } }
ParameterTypeRequiredDescription
window_titlestringYesWindow title to check

Use case: Verify login screens, confirm dialogs, or error modals display.

gui_execute_command

Execute a command in the terminal and capture a VNC screenshot.

{ "type": "gui_execute_command", "params": { "command": "weasis /opt/test-data/chest-xray.dcm", "wait_seconds": 5 } }
ParameterTypeRequiredDescription
commandstringYesCommand to execute
wait_secondsnumberNoSeconds to wait before screenshot (default: 2)

Use case: Visual regression testing, UI state validation. Automatically captures screenshot + metadata for FDA audit trail.

gui_application_process

Verify an application process is running.

{ "type": "gui_application_process", "params": { "process_name": "firefox" } }
ParameterTypeRequiredDescription
process_namestringYesProcess name to check

Use case: Ensure EDC client remains active, detect application crashes.

gui_screenshot_matches

Compare current screenshot against a reference image for visual regression testing.

{ "type": "gui_screenshot_matches", "params": { "reference_id": "ecrf_demographics_page", "threshold": 0.95, "ignore_regions": [[10, 10, 200, 30]], "match_area": [0, 0, 1920, 1080], "save_as_reference": false, "wait_before": 1.0 } }
ParameterTypeRequiredDescription
reference_idstringYesUnique identifier for baseline image
thresholdnumberNoSimilarity threshold 0.0-1.0 (default: 0.95)
ignore_regionsarrayNoList of [x, y, width, height] regions to mask
match_areaarrayNo[x, y, width, height] to crop both images
save_as_referencebooleanNoSave current screenshot as new baseline
wait_beforenumberNoSeconds to wait before capture (default: 1.0)

Use case: Detect UI regressions in EDC forms, verify DICOM viewer displays correctly, validate report layouts.

Storage: Reference images stored at {TEST_STORAGE_DIR}/reference_images/{reference_id}/ with metadata, diff images, and comparison history.

gui_click_element

Click a UI element at specified screen coordinates.

{ "type": "gui_click_element", "params": { "x": 600, "y": 400, "button": "left", "double_click": false, "wait_after": 0.5, "verify_click": true } }
ParameterTypeRequiredDescription
xnumberYesX coordinate
ynumberYesY coordinate
buttonstringNoMouse button: left, right, middle (default: left)
double_clickbooleanNoClick twice (default: false)
wait_afternumberNoSeconds to wait after click (default: 0.5)
verify_clickbooleanNoCapture verification screenshot (default: true)

Use case: Navigate EDC forms, click dialog buttons, interact with custom UI controls.

gui_form_fill

Fill form fields and optionally submit.

{ "type": "gui_form_fill", "params": { "fields": [ { "name": "patient_id", "value": "PT-001", "method": "tab" }, { "name": "visit_date", "value": "2024-01-15", "method": "click", "x": 300, "y": 250, "wait_after": 0.3 } ], "submit_button": {"x": 600, "y": 500}, "wait_after_submit": 1.0, "verify_submission": true } }
ParameterTypeRequiredDescription
fieldsarrayYesList of field objects (see below)
submit_buttonobjectNo{x, y} coordinates to click after filling
wait_after_submitnumberNoSeconds to wait after submit (default: 1.0)
verify_submissionbooleanNoCapture screenshot after submit (default: true)

Field object:

  • name - Field identifier (for logging)
  • value - Text to enter
  • method - tab (press Tab) or click (click coordinates)
  • x, y - Coordinates if method=click
  • wait_after - Seconds to wait after field (default: 0.3)

Use case: Automated data entry for eCRF forms, patient enrollment workflows, adverse event reporting.

gui_text_visible

Check if text is visible on screen using OCR.

Requires: pytesseract package and tesseract-ocr system package.

{ "type": "gui_text_visible", "params": { "text": "Consent Form Approved", "region": [100, 100, 800, 600], "case_sensitive": false, "lang": "eng", "wait_before": 1.0 } }
ParameterTypeRequiredDescription
textstringYesText to search for
regionarrayNo[x, y, width, height] to crop before OCR
case_sensitivebooleanNoCase-sensitive search (default: false)
langstringNoTesseract language (default: eng)
wait_beforenumberNoSeconds to wait before capture (default: 1.0)

Use case: Verify confirmation messages, validate report headers, check status indicators in legacy applications.

Installation:

pip install pytesseract sudo apt-get install tesseract-ocr # Ubuntu/Debian sudo dnf install tesseract # Fedora

Security Assertions

cis_benchmark

Run CIS benchmark checks.

{ "type": "cis_benchmark", "params": { "level": 1, "profile": "server", "controls": ["1.1.1.1", "5.2.1"] } }
ParameterTypeRequiredDescription
levelnumberNoCIS level (1 or 2)
profilestringNoserver or workstation
controlsstring[]NoSpecific controls to test

firewall_rule

Verify firewall rule exists.

{ "type": "firewall_rule", "params": { "action": "allow", "port": 22, "protocol": "tcp", "direction": "in" } }
ParameterTypeRequiredDescription
actionstringYesallow or deny
portnumberNoPort number
protocolstringNotcp or udp
directionstringNoin or out

Compound Assertions

all_of

All assertions must pass.

{ "type": "all_of", "params": { "assertions": [ { "type": "service_running", "params": { "service": "nginx" } }, { "type": "port_listening", "params": { "port": 80 } } ] } }

any_of

At least one assertion must pass.

{ "type": "any_of", "params": { "assertions": [ { "type": "file_exists", "params": { "path": "/etc/nginx/nginx.conf" } }, { "type": "file_exists", "params": { "path": "/etc/nginx/conf.d/default.conf" } } ] } }