Assertion Types
Complete reference for all test assertion types.
Overview
Assertions verify specific conditions in your built image. Each assertion has a type and parameters.
{
"type": "assertion_type",
"params": {
"key": "value"
}
}User Assertions
user_exists
Verify a user account exists.
{
"type": "user_exists",
"params": {
"username": "deploy",
"groups": ["docker", "sudo"],
"shell": "/bin/bash",
"home": "/home/deploy"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
username | string | Yes | Username to check |
groups | string[] | No | Required group memberships |
shell | string | No | Expected login shell |
home | string | No | Expected home directory |
group_exists
Verify a group exists.
{
"type": "group_exists",
"params": {
"name": "myapp",
"gid": 1001
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
name | string | Yes | Group name |
gid | number | No | Expected GID |
Service Assertions
service_running
Check a systemd service is active.
{
"type": "service_running",
"params": {
"service": "nginx"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
service | string | Yes | Service unit name |
service_enabled
Check a service is enabled at boot.
{
"type": "service_enabled",
"params": {
"service": "docker"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
service | string | Yes | Service unit name |
process_running
Check a process is running.
{
"type": "process_running",
"params": {
"name": "nginx",
"user": "www-data",
"count": 4
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
name | string | Yes | Process name |
user | string | No | Expected owner |
count | number | No | Expected process count |
Package Assertions
package_installed
Verify a package is installed.
{
"type": "package_installed",
"params": {
"package": "nginx",
"version": "1.24"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
package | string | Yes | Package name |
version | string | No | Version prefix to match |
package_not_installed
Verify a package is NOT installed.
{
"type": "package_not_installed",
"params": {
"package": "telnet"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
package | string | Yes | Package name |
Network Assertions
port_listening
Check a port is open.
{
"type": "port_listening",
"params": {
"port": 443,
"protocol": "tcp",
"address": "0.0.0.0"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
port | number | Yes | Port number |
protocol | string | No | tcp or udp (default: tcp) |
address | string | No | Bind address |
port_not_listening
Verify a port is NOT open.
{
"type": "port_not_listening",
"params": {
"port": 23
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
port | number | Yes | Port number |
protocol | string | No | tcp or udp |
http_responds
Verify an HTTP endpoint responds.
{
"type": "http_responds",
"params": {
"url": "http://localhost:8080/health",
"status": 200,
"contains": "healthy",
"method": "GET",
"timeout": 30,
"headers": {
"Authorization": "Bearer token"
}
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
url | string | Yes | URL to request |
status | number | No | Expected HTTP status (default: 200) |
contains | string | No | Response body must contain |
method | string | No | HTTP method (default: GET) |
timeout | number | No | Request timeout in seconds |
headers | object | No | Request headers |
dns_resolves
Verify DNS resolution works.
{
"type": "dns_resolves",
"params": {
"hostname": "example.com",
"record_type": "A"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
hostname | string | Yes | Hostname to resolve |
record_type | string | No | A, AAAA, MX, etc. |
File Assertions
file_exists
Check a file exists.
{
"type": "file_exists",
"params": {
"path": "/etc/myapp/config.yml",
"mode": "0644",
"owner": "root",
"group": "root"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
path | string | Yes | File path |
mode | string | No | Expected permissions |
owner | string | No | Expected owner |
group | string | No | Expected group |
file_not_exists
Verify a file does NOT exist.
{
"type": "file_not_exists",
"params": {
"path": "/etc/passwd-"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
path | string | Yes | File path |
file_contains
Check file contents.
{
"type": "file_contains",
"params": {
"path": "/etc/ssh/sshd_config",
"content": "PermitRootLogin no",
"regex": false
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
path | string | Yes | File path |
content | string | Yes | Content to find |
regex | boolean | No | Treat as regex |
directory_exists
Check a directory exists.
{
"type": "directory_exists",
"params": {
"path": "/var/log/myapp",
"mode": "0755"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
path | string | Yes | Directory path |
mode | string | No | Expected permissions |
mount_exists
Verify a mount point exists.
{
"type": "mount_exists",
"params": {
"path": "/data",
"fstype": "ext4",
"options": ["noexec"]
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
path | string | Yes | Mount point |
fstype | string | No | Filesystem type |
options | string[] | No | Mount options |
Command Assertions
command_output
Run a command and check output.
{
"type": "command_output",
"params": {
"command": "docker --version",
"contains": "Docker version",
"exit_code": 0,
"timeout": 30
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
command | string | Yes | Command to run |
contains | string | No | Output must contain |
exit_code | number | No | Expected exit code |
timeout | number | No | Timeout in seconds |
regex | boolean | No | Treat contains as regex |
command_succeeds
Verify command exits successfully.
{
"type": "command_succeeds",
"params": {
"command": "systemctl is-active nginx"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
command | string | Yes | Command to run |
timeout | number | No | Timeout in seconds |
GUI & Display Testing
gui_application_opens
Launch a GUI application and verify its window appears.
{
"type": "gui_application_opens",
"params": {
"command": "firefox",
"window_title": "Firefox",
"timeout": 10
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
command | string | Yes | Command to launch application |
window_title | string | Yes | Expected window title |
timeout | number | No | Seconds to wait for window (default: 10) |
Use case: Verify EDC clients, imaging viewers, or CTMS interfaces launch successfully.
gui_window_visible
Check if a window with specific title is visible on screen.
{
"type": "gui_window_visible",
"params": {
"window_title": "EDC Login"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
window_title | string | Yes | Window title to check |
Use case: Verify login screens, confirm dialogs, or error modals display.
gui_execute_command
Execute a command in the terminal and capture a VNC screenshot.
{
"type": "gui_execute_command",
"params": {
"command": "weasis /opt/test-data/chest-xray.dcm",
"wait_seconds": 5
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
command | string | Yes | Command to execute |
wait_seconds | number | No | Seconds to wait before screenshot (default: 2) |
Use case: Visual regression testing, UI state validation. Automatically captures screenshot + metadata for FDA audit trail.
gui_application_process
Verify an application process is running.
{
"type": "gui_application_process",
"params": {
"process_name": "firefox"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
process_name | string | Yes | Process name to check |
Use case: Ensure EDC client remains active, detect application crashes.
gui_screenshot_matches
Compare current screenshot against a reference image for visual regression testing.
{
"type": "gui_screenshot_matches",
"params": {
"reference_id": "ecrf_demographics_page",
"threshold": 0.95,
"ignore_regions": [[10, 10, 200, 30]],
"match_area": [0, 0, 1920, 1080],
"save_as_reference": false,
"wait_before": 1.0
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
reference_id | string | Yes | Unique identifier for baseline image |
threshold | number | No | Similarity threshold 0.0-1.0 (default: 0.95) |
ignore_regions | array | No | List of [x, y, width, height] regions to mask |
match_area | array | No | [x, y, width, height] to crop both images |
save_as_reference | boolean | No | Save current screenshot as new baseline |
wait_before | number | No | Seconds to wait before capture (default: 1.0) |
Use case: Detect UI regressions in EDC forms, verify DICOM viewer displays correctly, validate report layouts.
Storage: Reference images stored at {TEST_STORAGE_DIR}/reference_images/{reference_id}/ with metadata, diff images, and comparison history.
gui_click_element
Click a UI element at specified screen coordinates.
{
"type": "gui_click_element",
"params": {
"x": 600,
"y": 400,
"button": "left",
"double_click": false,
"wait_after": 0.5,
"verify_click": true
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
x | number | Yes | X coordinate |
y | number | Yes | Y coordinate |
button | string | No | Mouse button: left, right, middle (default: left) |
double_click | boolean | No | Click twice (default: false) |
wait_after | number | No | Seconds to wait after click (default: 0.5) |
verify_click | boolean | No | Capture verification screenshot (default: true) |
Use case: Navigate EDC forms, click dialog buttons, interact with custom UI controls.
gui_form_fill
Fill form fields and optionally submit.
{
"type": "gui_form_fill",
"params": {
"fields": [
{
"name": "patient_id",
"value": "PT-001",
"method": "tab"
},
{
"name": "visit_date",
"value": "2024-01-15",
"method": "click",
"x": 300,
"y": 250,
"wait_after": 0.3
}
],
"submit_button": {"x": 600, "y": 500},
"wait_after_submit": 1.0,
"verify_submission": true
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
fields | array | Yes | List of field objects (see below) |
submit_button | object | No | {x, y} coordinates to click after filling |
wait_after_submit | number | No | Seconds to wait after submit (default: 1.0) |
verify_submission | boolean | No | Capture screenshot after submit (default: true) |
Field object:
name- Field identifier (for logging)value- Text to entermethod-tab(press Tab) orclick(click coordinates)x,y- Coordinates if method=clickwait_after- Seconds to wait after field (default: 0.3)
Use case: Automated data entry for eCRF forms, patient enrollment workflows, adverse event reporting.
gui_text_visible
Check if text is visible on screen using OCR.
Requires: pytesseract package and tesseract-ocr system package.
{
"type": "gui_text_visible",
"params": {
"text": "Consent Form Approved",
"region": [100, 100, 800, 600],
"case_sensitive": false,
"lang": "eng",
"wait_before": 1.0
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
text | string | Yes | Text to search for |
region | array | No | [x, y, width, height] to crop before OCR |
case_sensitive | boolean | No | Case-sensitive search (default: false) |
lang | string | No | Tesseract language (default: eng) |
wait_before | number | No | Seconds to wait before capture (default: 1.0) |
Use case: Verify confirmation messages, validate report headers, check status indicators in legacy applications.
Installation:
pip install pytesseract
sudo apt-get install tesseract-ocr # Ubuntu/Debian
sudo dnf install tesseract # FedoraSecurity Assertions
cis_benchmark
Run CIS benchmark checks.
{
"type": "cis_benchmark",
"params": {
"level": 1,
"profile": "server",
"controls": ["1.1.1.1", "5.2.1"]
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
level | number | No | CIS level (1 or 2) |
profile | string | No | server or workstation |
controls | string[] | No | Specific controls to test |
firewall_rule
Verify firewall rule exists.
{
"type": "firewall_rule",
"params": {
"action": "allow",
"port": 22,
"protocol": "tcp",
"direction": "in"
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
action | string | Yes | allow or deny |
port | number | No | Port number |
protocol | string | No | tcp or udp |
direction | string | No | in or out |
Compound Assertions
all_of
All assertions must pass.
{
"type": "all_of",
"params": {
"assertions": [
{ "type": "service_running", "params": { "service": "nginx" } },
{ "type": "port_listening", "params": { "port": 80 } }
]
}
}any_of
At least one assertion must pass.
{
"type": "any_of",
"params": {
"assertions": [
{ "type": "file_exists", "params": { "path": "/etc/nginx/nginx.conf" } },
{ "type": "file_exists", "params": { "path": "/etc/nginx/conf.d/default.conf" } }
]
}
}