Skip to Content
TestingDefault Tests

Default Tests

Every OpenFactory build runs a standard set of tests automatically. These verify core functionality without any configuration.

Boot Verification

The first and most critical test.

What’s Tested:

  • GRUB bootloader loads
  • Kernel initializes
  • systemd starts services
  • Login prompt appears

Pass Criteria:

  • System reaches multi-user target within timeout
  • No kernel panics or critical errors
  • GDM starts (for desktop images)

Failure Modes:

  • Missing bootloader configuration
  • Kernel module errors
  • Init system failures
  • Filesystem mount issues

Login Test

Verifies user authentication works.

What’s Tested:

  • SSH key authentication (if configured)
  • Password authentication (if enabled)
  • User shell access
  • Home directory creation

Pass Criteria:

  • Can authenticate as configured user
  • Shell prompt received
  • Basic commands execute

Failure Modes:

  • User not created
  • Incorrect permissions
  • PAM configuration errors
  • SSH service not running

Package Installation Check

Verifies requested packages are installed.

What’s Tested:

  • All explicitly requested packages
  • Feature-related packages
  • Critical system packages

Pass Criteria:

  • Package manager reports installed
  • Binaries exist in PATH
  • Libraries are loadable

Example:

Checking: docker-ce, docker-compose, openssh-server, ufw
✓ docker-ce: installed (24.0.7)
✓ docker-compose: installed (2.23.0)
✓ openssh-server: installed (9.2p1)
✓ ufw: installed (0.36.1)

Network Connectivity

Basic network verification.

What’s Tested:

  • Network interface configuration
  • DHCP client (if configured)
  • DNS resolution
  • External connectivity

Pass Criteria:

  • Interface has IP address
  • Can ping gateway
  • Can resolve DNS names
  • Can reach external hosts

Test Commands:

ip addr show
ping -c 1 gateway
nslookup example.com
curl -I https://example.com

Service Health

Verifies enabled services are running.

What’s Tested:

  • SSH service (if enabled)
  • Docker daemon (if enabled)
  • Desktop manager (if desktop)
  • Custom services

Pass Criteria:

  • systemd reports active
  • Listening on expected ports
  • No failed dependencies

Example Output:

✓ ssh.service: active (running)
✓ docker.service: active (running)
✓ ufw.service: active (exited)

Firewall Status

If firewall feature is enabled.

What’s Tested:

  • UFW is active
  • Default policy is deny
  • Expected ports are open

Pass Criteria:

  • Firewall daemon running
  • Rules match configuration
  • Can connect to allowed ports

Test Timeouts

Default timeouts for each test phase:

PhaseTimeout
VM Boot5 minutes
Login2 minutes
Package check1 minute
Network test1 minute
Service check1 minute

Skipping Default Tests

In rare cases, you may need to skip certain default tests:

Build the image but skip the network connectivity test
(this is an air-gapped system)

OpenFactory will note which tests were skipped in the results.

Test Logs

All test output is captured:

  1. Build Details → Tests tab
  2. View stdout/stderr for each test
  3. Download full test logs

Common Failures

Boot Failure

✗ Boot verification failed
  Timeout waiting for login prompt

Likely Causes:

  • Kernel configuration issue
  • Missing initramfs modules
  • Disk configuration problem

Package Not Found

✗ Package check failed
  Package 'docker-ce' not installed

Likely Causes:

  • Repository not enabled
  • Package name incorrect
  • Dependency conflict

Service Not Running

✗ Service check failed
  ssh.service: inactive (dead)

Likely Causes:

  • Service not enabled
  • Configuration error
  • Missing dependency

Retry Behavior

Tests include automatic retries:

  • Network tests: 3 retries with backoff
  • Service checks: Wait up to 30s for startup
  • Login attempts: 3 tries

This handles timing-related transient failures.

Custom AssertionsVM Management