Користувацькі assertions

Custom assertions перетворюють requirement на command, який VM harness може execute. Use for behavior small default suite does not cover.

Structure

Each custom-test group needs description and one or more structured assertions:

{
  "description": "Verify the web service",
  "category": "application",
  "assertions": [
    {
      "type": "service_running",
      "description": "Nginx is active",
      "params": { "service": "nginx" },
      "expected_visual": "The evidence panel shows nginx active"
    },
    {
      "type": "http_responds",
      "description": "Local health endpoint responds",
      "params": { "url": "http://localhost/health", "status": 200 }
    }
  ]
}

description required both on group and each assertion. on_vm selects VM ID in multi-VM test environment; without it runner uses primary. expected_visual supplies intended screenshot state when visual evidence gate active.

Design assertions around outcomes

• Check service with service_running, not merely package exists.
• Check local endpoint before testing external route.
• Use file_contains for one stable configuration fact, not complete-file snapshot that breaks on harmless formatting.
• Use command_succeeds only with deterministic, non-interactive commands.
• Give long commands timeout_seconds only when needed. Executor clamps command timeouts to 1–1,800 seconds.
• Never put credentials in commands, descriptions, expected output, URLs; these fields can appear in evidence and logs.

Failure semantics

Unknown type currently accepted by recipe model but becomes error when executor cannot find handler. Known assertions omitting required parameters can be dropped during test-plan assembly with warning. Validate recipe and inspect materialized test plan before starting build.

If on_vm names VM not present, assertion is skipped. Do not treat skipped or error as successful verification.

Multi-VM example

{
  "description": "Client reaches the API node",
  "assertions": [
    {
      "type": "http_responds",
      "description": "API health is reachable from the client",
      "on_vm": "client",
      "params": { "url": "http://api:8080/health", "status": 200 }
    }
  ],
  "environment": {
    "vms": [
      { "vm_id": "client", "role": "client", "networks": ["lan"] },
      { "vm_id": "api", "role": "server", "networks": ["lan"] }
    ],
    "networks": [
      { "network_id": "lan", "type": "isolated", "dhcp": true }
    ]
  }
}

VM-name resolution depends on test topology and discovered addresses. Confirm rendered topology and assertion resolved command in run evidence.

For every supported type and parameter see Assertion types.