Testing

Every OpenFactory build is automatically tested in a KVM virtual machine before the ISO is made available for download.

Why Automated Testing?

Building a custom Linux image involves many components that must work together:

• Kernel and bootloader configuration
• Service dependencies and startup order
• Network configuration
• User authentication
• Package compatibility

OpenFactory’s testing system verifies all these components automatically.

Test Infrastructure

┌─────────────────────────────────────────────────────────┐
│                    Test Pipeline                         │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  ┌──────────┐    ┌──────────┐    ┌──────────────────┐  │
│  │   ISO    │───▶│   KVM    │───▶│   Test Runner    │  │
│  │  Image   │    │   VM     │    │   (assertions)   │  │
│  └──────────┘    └──────────┘    └──────────────────┘  │
│                       │                    │            │
│                       │              ┌─────▼─────┐     │
│                       │              │  Results  │     │
│                       └─────────────▶│  Report   │     │
│                                      └───────────┘     │
│                                                          │
└─────────────────────────────────────────────────────────┘

Test Categories

Default Tests

Every build automatically runs:

• Boot verification - System boots successfully
• Login test - Users can authenticate
• Package check - Requested packages installed
• Network test - Basic connectivity works

Custom Assertions

Add your own verification tests:

• Service running checks
• Port listening verification
• HTTP endpoint testing
• File content validation
• Command output checking

CIS Benchmarks

Security compliance testing:

• CIS Debian Linux Benchmark
• CIS Ubuntu Linux Benchmark
• Configurable hardening levels

Test Results

After testing completes, you can view:

• Pass/Fail status for each test
• Detailed logs from test execution
• Screenshots of the VM during testing
• Timing information for each stage

Quick Links

Test Failures

When tests fail:

1. Review the logs - Detailed output shows what went wrong
2. Check configuration - Verify your build recipe
3. Modify and rebuild - Make changes and try again
4. Ask for help - OpenFactory can suggest fixes

Failed builds provide detailed diagnostics to help identify and fix issues.