Evidence-oriented testing for regulated research systems

OpenFactory can automate bounded infrastructure and application checks for a contract research organization. It does not determine FDA compliance, validate an entire computerized system automatically, or replace the regulated organization’s quality system and accountable approvals.

21 CFR Part 11  applies to specified electronic records and signatures and includes controls beyond an operating-system image: system validation, record protection and retrieval, access and authority checks, time-stamped audit trails, training, policies, documentation controls, and signature requirements. The applicable predicate rules and intended use must be established by qualified legal, regulatory, and quality personnel.

FDA’s current Computer Software Assurance guidance  describes a risk-based approach for production and quality-management-system software. Teams should determine applicability to their system and context rather than treating a generic test count as validation.

What OpenFactory evidence can support

• exact recipe and source provenance for an image build;
• package, service, file, port, and command assertions in a booted guest;
• bounded GUI and screenshot evidence where a supported assertion runs;
• per-run timestamps, output, and artifact identifiers;
• repeatable negative and recovery tests; and
• comparison of a new artifact against an approved baseline.

Each item is evidence for a stated requirement. None establishes regulatory acceptability on its own.

Start with intended use and risk

Before authoring tests, document:

1. the intended use of the computerized system;
2. regulated records and signatures, if any;
3. users, roles, interfaces, and data flows;
4. patient, product-quality, and data-integrity risks;
5. requirements and acceptance criteria tied to those risks;
6. supplier and component responsibilities; and
7. change, incident, backup, recovery, retention, and decommissioning procedures.

The operating-system image is only one configuration item within that system.

Example bounded scenario

This example checks a synthetic integration service and local audit configuration. It does not claim that an EDC, LIMS, safety database, or Part 11 workflow is validated.

{
  "id": "synthetic-integration-smoke",
  "name": "Synthetic integration and audit smoke test",
  "enabled": true,
  "tests": ["boot", "login", "packages", "services"],
  "custom_tests": [
    {
      "description": "Confirm the synthetic receiver and audit controls are present.",
      "assertions": [
        {
          "type": "service_running",
          "description": "The synthetic receiver is running.",
          "params": {"service": "synthetic-receiver"}
        },
        {
          "type": "port_listening",
          "description": "The synthetic receiver listens on its lab port.",
          "params": {"port": 2575}
        },
        {
          "type": "service_running",
          "description": "The Linux audit daemon is running.",
          "params": {"service": "auditd"}
        },
        {
          "type": "file_contains",
          "description": "The approved synthetic data path has an audit watch.",
          "params": {
            "path": "/etc/audit/rules.d/research-system.rules",
            "content": "-w /var/lib/synthetic-study"
          }
        }
      ]
    }
  ]
}

Use only synthetic, non-sensitive test data in shared build and screenshot systems. A screenshot can expose subject identifiers, credentials, notifications, or unrelated desktop content. Define capture, redaction, access, retention, export, and deletion controls before collecting it.

Evidence packet

For each approved test run, preserve:

• requirement and risk identifiers;
• test protocol and expected result;
• recipe revision, source commits, dependency inventory, and image digest;
• environment and test-data identity;
• raw output, screenshots where justified, timestamps, and runner version;
• deviations, failed steps, investigations, and retest links;
• reviewer identity, decision, and date; and
• traceability from requirement to evidence and release decision.

Do not relabel an automated event log or screenshot folder as a Part 11 audit trail. Part 11 audit-trail controls concern regulated record actions, retention, availability, and integrity across the applicable system.

Change and regression workflow

1. Assess the impact of the proposed change.
2. Select tests based on risk and affected requirements.
3. Build a new immutable artifact; retain the prior approved artifact.
4. Run the protocol in a controlled environment.
5. Review failures and deviations without deleting unfavorable evidence.
6. Obtain the required quality, security, business, and regulatory approvals.
7. Deploy through change control and verify the production configuration.
8. Monitor for drift and execute the tested recovery path when needed.

OpenFactory can shorten evidence collection for steps it actually automates. The regulated organization remains responsible for intended-use validation, procedural controls, data governance, and the final release decision.