Production Deployment

This guide covers hardening a self-hosted OpenFactory instance for production use.

Reverse Proxy with Nginx

Use Nginx to terminate SSL and proxy to the backend services.


server {
    listen 443 ssl http2;
    server_name openfactory.example.com;
 
    ssl_certificate /etc/letsencrypt/live/openfactory.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/openfactory.example.com/privkey.pem;
 
    # Frontend
    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
 
    # Backend API
    location /api/ {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 300s;  # builds can take several minutes
    }
 
    # SSE (Server-Sent Events) for build status
    location /api/stream/ {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_buffering off;
        proxy_cache off;
        proxy_read_timeout 3600s;
    }
}
 
server {
    listen 80;
    server_name openfactory.example.com;
    return 301 https://$host$request_uri;
}

Update NEXTAUTH_URL, FRONTEND_URL, and ALLOWED_ORIGINS to use your domain with HTTPS.

systemd Service Units

Replace screen sessions with systemd for automatic restarts and logging.

elster-terminal-backend


# /etc/systemd/system/openfactory-backend.service
[Unit]
Description=OpenFactory Backend
After=network.target
 
[Service]
Type=simple
User=openfactory
WorkingDirectory=/opt/openfactory/elster-terminal-backend
EnvironmentFile=/opt/openfactory/elster-terminal-backend/.env
ExecStart=/opt/openfactory/elster-terminal-backend/venv/bin/uvicorn \
    app.main:app --host 0.0.0.0 --port 8000 --workers 4 \
    --timeout-keep-alive 30 --timeout-notify 120
Restart=always
RestartSec=5
 
[Install]
WantedBy=multi-user.target

cto-gui-libvirt-backend


# /etc/systemd/system/openfactory-libvirt.service
[Unit]
Description=OpenFactory Libvirt Backend
After=network.target libvirtd.service
 
[Service]
Type=simple
User=openfactory
WorkingDirectory=/opt/openfactory/cto-gui-libvirt-backend
EnvironmentFile=/opt/openfactory/cto-gui-libvirt-backend/.env
ExecStart=/opt/openfactory/cto-gui-libvirt-backend/venv/bin/uvicorn \
    app.main:app --host 0.0.0.0 --port 8001 --workers 2
Restart=always
RestartSec=5
 
[Install]
WantedBy=multi-user.target

web-terminal-backend


# /etc/systemd/system/openfactory-terminal.service
[Unit]
Description=OpenFactory Web Terminal
After=network.target
 
[Service]
Type=simple
User=openfactory
WorkingDirectory=/opt/openfactory/web-terminal-backend
EnvironmentFile=/opt/openfactory/web-terminal-backend/.env
ExecStart=/opt/openfactory/web-terminal-backend/venv/bin/uvicorn \
    app.main:app --host 0.0.0.0 --port 8002 --workers 2
Restart=always
RestartSec=5
 
[Install]
WantedBy=multi-user.target

Enable and start:


sudo systemctl daemon-reload
sudo systemctl enable --now openfactory-backend openfactory-libvirt openfactory-terminal

Firewall Rules

Only expose the Nginx ports publicly. Backend ports should only be accessible from localhost:


# UFW example
sudo ufw allow 22/tcp    # SSH
sudo ufw allow 80/tcp    # HTTP (redirect to HTTPS)
sudo ufw allow 443/tcp   # HTTPS
sudo ufw enable

Backend ports (8000, 8001, 8002) should not be exposed — Nginx proxies all traffic.

Backup Strategy

What to Back Up

Path	Contents	Frequency
`elster-terminal-backend/data/`	User accounts, builds, conversations, logs	Daily
`elster-terminal-backend/data/recipes/`	Build recipe definitions	Daily
`cto-gui-libvirt-backend/data/`	Test results, benchmarks, VM metadata	Weekly
`web-terminal-backend/data/`	SSH host configs and keys	On change
All `.env` files	Service configuration	On change

Example Backup Script


#!/bin/bash
BACKUP_DIR="/backups/openfactory/$(date +%Y-%m-%d)"
mkdir -p "$BACKUP_DIR"
 
tar czf "$BACKUP_DIR/backend-data.tar.gz" \
    /opt/openfactory/elster-terminal-backend/data/
 
tar czf "$BACKUP_DIR/libvirt-data.tar.gz" \
    /opt/openfactory/cto-gui-libvirt-backend/data/
 
tar czf "$BACKUP_DIR/terminal-data.tar.gz" \
    /opt/openfactory/web-terminal-backend/data/
 
# Keep 30 days of backups
find /backups/openfactory -maxdepth 1 -mtime +30 -exec rm -rf {} +

Monitoring

Service Health

Monitor the health endpoints with your preferred tool (Prometheus, Uptime Kuma, etc.):

http://localhost:8000/api/health — backend
http://localhost:8001/api/health — libvirt backend
http://localhost:3000 — frontend

Disk Space

Build ISOs can be large (1-4 GB each). Monitor disk usage on the builds directory and set up automatic cleanup of old builds.

systemd Journal


# View logs for a specific service
journalctl -u openfactory-backend -f
 
# View all OpenFactory logs
journalctl -u 'openfactory-*' --since "1 hour ago"

Updating

To update to a newer version:


cd /opt/openfactory
git pull origin main
 
# Reinstall dependencies
cd elster-terminal-backend && source venv/bin/activate && pip install -r requirements.txt && deactivate
cd ../cto-gui-libvirt-backend && source venv/bin/activate && pip install -r requirements.txt && deactivate
cd ../web-terminal-backend && source venv/bin/activate && pip install -r requirements.txt && deactivate
cd ../elster-terminal && npm install
 
# Restart services
sudo systemctl restart openfactory-backend openfactory-libvirt openfactory-terminal