MCP Integration

OpenFactory supports the Model Context Protocol (MCP) , an open standard that lets AI assistants like Claude, Cursor, and other MCP-compatible clients interact directly with OpenFactory. Build OS images, manage VMs, run tests, and browse recipes — all through natural language.

Quick Start

Claude Desktop

Add this to your Claude Desktop configuration (~/.claude/claude_desktop_config.json):

{
  "mcpServers": {
    "openfactory": {
      "url": "https://console.openfactory.tech/api/mcp/sse"
    }
  }
}

Cursor

Add this to your Cursor MCP settings (.cursor/mcp.json):

{
  "mcpServers": {
    "openfactory": {
      "url": "https://console.openfactory.tech/api/mcp/sse"
    }
  }
}

Claude Code

claude mcp add openfactory \
  --transport sse \
  --url "https://console.openfactory.tech/api/mcp/sse"

Claude.ai (Browser)

You can also connect OpenFactory directly in the Claude.ai web app:

1. Open claude.ai  and go to Settings
2. Navigate to Customize → Connectors
3. Click the + button to add a new connector
4. Enter the SSE endpoint: https://console.openfactory.tech/api/mcp/sse
5. Name it OpenFactory and save

Once connected, you’ll see OpenFactory’s tools listed under the connector. Set tool permissions to Needs approval or Always allow based on your preference.

Restart your client and you’ll see OpenFactory’s tools available.

Connection

The MCP server uses the SSE transport. Your MCP client connects to the SSE endpoint for server-to-client messages and posts to the messages endpoint for client-to-server messages. Most MCP clients handle this automatically.

Your session is tied to your IP address, so your builds, VMs, and test results persist across requests.

Available Tools

OpenFactory exposes 29 tools across 5 categories.

Build Tools

Recipe Tools

VM Tools

Desktop Tools (Agent Desktop Interaction)

These tools are designed for AI agents running inside OpenFactory’s Direktor agent containers. Each agent has an assigned desktop VM (XFCE + Firefox) that it can control. Every tool combines the VM Interaction API with OmniParser  — returning both a screenshot and a structured list of all detected UI elements (buttons, text, icons) with pixel coordinates. No need to call screenshot and parse separately.

Each tool returns:

• A screenshot image (PNG) of the desktop after the action
• A parsed elements list with center: [x, y], content, type, and interactivity for each detected element

Test Tools

Examples

Once connected, you can interact with OpenFactory using natural language:

Building an image:

“Build me a Debian Bookworm server with SSH, Docker, and monitoring enabled”

Checking build status:

“Show my recent builds” or “What’s the status of my latest build?”

Running tests:

“Run tests on build abc123” or “Show the test results for my last build”

Working with recipes:

“Show me the available healthcare recipes” or “Create a build from the GxP MedTech template with extra packages: vim, htop”

Managing VMs:

“Create a VM from my latest build” or “List my running VMs”

Interacting with a VM GUI:

“Take a screenshot of my VM” or “Click on the login button at coordinates 500, 300”

GUI Interaction with interact_vm

The interact_vm tool lets you control a VM’s graphical display via VNC. Pass a JSON array of actions that execute sequentially:

[
  {"type": "click", "x": 500, "y": 300},
  {"type": "wait", "duration": 1},
  {"type": "type", "text": "hello world"},
  {"type": "key", "key": "Return"},
  {"type": "screenshot"}
]

Supported action types:

By default, screenshot_after: true returns a screenshot image after all actions complete. Use screenshot_vm for a standalone screenshot without any actions.

Desktop Interaction with desktop_* Tools

The desktop tools are designed for Direktor agents with an assigned desktop VM. Unlike interact_vm which returns raw screenshots, desktop tools automatically run OmniParser  on every screenshot to detect all UI elements and their coordinates.

Typical workflow:

1. desktop_screenshot(vm_name="of-desktop-abc-myorg-def123")
   → Returns screenshot + elements: [{"content": "Firefox", "center": [512, 400], "type": "icon"}, ...]

2. desktop_click(vm_name="of-desktop-abc-myorg-def123", x=512, y=400)
   → Clicks Firefox icon, returns new screenshot + updated elements

3. desktop_open_url(vm_name="of-desktop-abc-myorg-def123", url="https://example.com")
   → Opens URL in Firefox, waits for load, returns screenshot + page elements

Opening a URL directly:

desktop_open_url(vm_name="of-desktop-...", url="https://github.com")

This sends Ctrl+L → types URL → Enter → waits 5 seconds → screenshot + parse. Returns both the visual screenshot and all detected page elements with coordinates.

Element response format:

{
  "elements": [
    {
      "center": [512, 300],
      "bbox": [480, 280, 544, 320],
      "content": "Sign In",
      "type": "button",
      "interactivity": true
    }
  ],
  "total": 47,
  "interactive_count": 12,
  "summary": "Found 47 elements (12 interactive):\n  * [button] \"Sign In\" at (512, 300)\n  ..."
}

Sign In

If you create an account at console.openfactory.tech , you can link your MCP client to it with an API key. This lets you access builds from the web console and keeps your work consistent across different networks and devices.

Creating an API Key

1. Sign in at console.openfactory.tech 
2. Go to Settings and generate an MCP API key
3. Add the key to your client config:

Claude Desktop / Cursor:

{
  "mcpServers": {
    "openfactory": {
      "url": "https://console.openfactory.tech/api/mcp/sse",
      "headers": {
        "Authorization": "Bearer of_mcp_your_key_here"
      }
    }
  }
}

Claude Code:

claude mcp remove openfactory
claude mcp add openfactory \
  --transport sse \
  --url "https://console.openfactory.tech/api/mcp/sse" \
  --header "Authorization: Bearer of_mcp_your_key_here"

Managing API Keys

You can also manage keys via the API:

# Create a key
POST /api/mcp-keys
Authorization: Bearer <your_jwt_token>
Content-Type: application/json
 
{"name": "Claude Desktop"}

# List keys
GET /api/mcp-keys
Authorization: Bearer <your_jwt_token>

# Revoke a key
DELETE /api/mcp-keys/{key_id}
Authorization: Bearer <your_jwt_token>

API keys are prefixed with of_mcp_ and shown only once when created. Revoking a key immediately disconnects any clients using it.

Troubleshooting

Connection refused or timeout

Test the connection with:

curl -H "Accept: text/event-stream" \
  https://console.openfactory.tech/api/mcp/sse

You should see SSE events streaming back. If not, the service may be temporarily unavailable.

Tools not appearing

Restart your MCP client after adding the server configuration. Claude Desktop and Cursor may need a full restart to pick up new MCP servers.

”Invalid API key” error

If you’re using an API key, verify it hasn’t been revoked. Remove the Authorization header to fall back to your default session, or generate a new key from the console.

Build or VM operations failing

Check that the operation is valid (e.g., you can only download ISOs from completed builds).