How to Drive Backstage Headlessly with the Automation API

How to Drive Backstage Headlessly with the Automation API

Use the Automation API to control a running Backstage instance from scripts, integration tools, or an AI agent over HTTP — without touching the screen. You can import media, build projection surfaces, control sequences, read live state, and record repeatable sessions, all through REST calls or via a native MCP (Model Context Protocol) endpoint.

[USER-ACTION-REQUIRED: Attach a screenshot of the Settings > Remote Control section showing the Agent / Automation API controls here]

Before you start

• Backstage is running and a project is loaded (or an empty project is open).
• The built-in REST/HTTP server is active (Settings > REST Server > Active — enabled by default).
• You have network access to the Backstage server on port 8080 (HTTP) or 8443 (HTTPS).
• You have a tool that can send HTTP requests — a command-line HTTP client, PowerShell, a Python HTTP library, or similar. For MCP access, an MCP-compatible client (AI agent, IDE extension, or custom integration).

Steps

1. Enable the Automation API

Open Settings > Remote Control. Tick Enable Agent / Automation API. The change takes effect immediately — no restart required. This single gate controls both the REST endpoints (/api/automation/*) and the MCP endpoint (/mcp).

2. Secure the API for production use

The API is designed for closed, controlled networks. Before enabling it on a show machine, work through this checklist:

1. Change the default credentials. The REST server ships with a default administrator username and password. Set a strong username and password in the REST credentials (Basic auth) fields. Backstage shows a red warning while the defaults are unchanged.

2. Set an IP allowlist. In the Allowed IPs/CIDRs field, enter the IP addresses or CIDR ranges of machines that should have access (comma-separated). Leave it empty only if every host on the network is trusted. Requests from non-listed IPs receive a 404 — the API surface is hidden, not just refused.

3. Disable capability sets you do not need. Four feature toggles let you shrink what the API can do. Untick any your client does not require (see Options explained below).

4. Restrict file access to sandbox roots. If your client imports or saves files, set the file sandbox root to a semicolon-separated list of allowed directories. File-path actions that resolve outside every listed root are rejected with a 400 error.

5. Use a dedicated service account. Authenticate the API or MCP client as a dedicated BSM user account — do not reuse a human operator's login. A shared identity muddies the audit trail and risks session collisions if the same account drives a Remote UI session concurrently.

3. Verify the connection

From your client machine, send a GET request to /api/automation/actions to confirm the API is reachable. A successful response returns a JSON object with an actions array listing every registered action name. If you get a 404, the API is not enabled. If you get a 401, check your credentials.

4. Read live state

Send a GET request to /api/automation/state/ followed by a category to see what the server currently has loaded. The categories are scene (surfaces, projectors, groups), assets (imported media), outputs (GPU, NDI, recording), sequencer (sequences and transport state), and project (project metadata). State reads are never capability-gated — they always work when the API is enabled.

5. Execute an action

Send a POST request to /api/automation/action to change the scene. For example, run the create_scene_object action with a plane type and a name such as "Front Screen". A successful response reports a success flag and a result string such as "Created plane 'Front Screen' (id 42)". The id in the result string is the scene-node ID you can use in subsequent actions (transforms, assignments, deletions) and that appears in /state/scene.

6. Import an asset and assign it

Import a media file and assign it to the surface you just created:

1. POST the import_asset action with the media file path. It is fully synchronous — it returns only once the asset exists in Backstage, and its result includes the new asset id.

2. POST the assign_asset_to_surface action with the asset and the target object (for example, the "Front Screen" surface). This creates a sequence and starts playback, so you can verify it by reading /state/sequencer and looking for a playing transport state.

7. Discover action schemas

To see every available action with its full parameter schema, send a GET request to /api/automation/actions with the verbose query flag. This returns each action's name, description, category, and a params array with every parameter's name, type, description, and whether it is required. Use this to drive actions programmatically without guessing argument names.

8. Record and replay a session

You can record a sequence of API calls and replay them later — useful for repeatable setup routines or testing.

1. Start recording: POST to /api/automation/record/start. The response confirms recording has started.

2. Perform your actions. Send action and UI-input calls as normal. Only accepted (non-error) calls are captured. State reads are not recorded.

3. Stop recording: POST to /api/automation/record/stop. The response returns the full session as JSON.

4. Replay: POST the session to /api/automation/replay. Each step is re-dispatched against the current state. By default, replay stops at the first failing step; a continue-on-error flag runs all steps regardless.

The replay response reports how many steps ran, how many failed, and a per-step result list so you can identify exactly which step failed and why.

Options explained

Endpoint reference

Status codes:200 success (check the ok field in the body for per-action detail), 400 bad request or sandbox violation, 401 authentication failure, 403 capability disabled or insufficient role (see Role-based access), 404 API not enabled or IP not allowlisted, 503 no UI available, 504 action timeout.

Native MCP endpoint

Backstage also exposes the Automation API as a native MCP (Model Context Protocol) endpoint. This lets any MCP-compatible client — an AI agent, an IDE extension, or a custom integration — drive the server directly using the standard MCP wire format, without writing raw REST calls.

The MCP endpoint is available at /mcp over Streamable HTTP (JSON-RPC 2.0). It maps the same action catalog and state model onto MCP primitives:

Connecting an MCP client

Point your MCP client at the Backstage server's HTTP address with the /mcp path (for example, the server host and port followed by /mcp). The MCP endpoint shares the same enablement gate (Enable Agent / Automation API), HTTP Basic authentication, IP allowlist, role-based access, and capability toggles as the REST surface. If you have already configured the REST API, the MCP endpoint works with the same credentials and security posture — no additional setup is needed.

Role-based access

Every request — whether it arrives via REST or MCP — is authorized against the caller's BSM user role. Three access tiers apply:

If the authenticated user's role is below the required tier, the request returns HTTP 403. For example, a user account with the User role can read state and list actions but cannot create objects or import media.

Common Mistakes

• Every automation endpoint returns 404 — the API is disabled (default in production builds). Enable it in Settings > Remote Control > Enable Agent / Automation API. A 404 also appears if the caller's IP is not on the allowlist.

• 401 on every request — wrong credentials. The API uses HTTP Basic authentication. Make sure your client sends the correct username and password set in Settings > Remote Control.

• 403 "automation capability disabled" — the specific capability set your action needs is turned off. Check the feature toggles in Settings > Remote Control (Mutations, File access, UI input, or Record/replay).

• 403 on mutations but reads work fine — the authenticated account's BSM role is too low. Reads require only the User role; mutations require Operator, and file-path actions require Administrator. Assign the appropriate role to the account in BSM user management.

• 400 "path not allowed" on import or save — the file path resolves outside the sandbox roots. Add the directory to the file sandbox root, or move the file into an already-allowed directory.

• Replay fails on steps that worked during recording — the session was recorded against a different project state. Sessions use widget locators (text labels, not pixel coordinates), so they survive UI rescaling, but the underlying objects must exist. Load the same project or rebuild the expected state before replaying.

Tips

• Prefer named actions over UI injection. The action endpoint with a semantic action (such as create_scene_object or set_position) is more reliable than clicking widgets through the UI-input endpoints. Use UI injection only for operations that have no corresponding action.

• Actions accept names or IDs. You can address objects, assets, and nodes by their display name or by the numeric ID that the state endpoints return. IDs are unambiguous; names are convenient for one-off scripts.

• Created IDs come back in the result string. Actions like create_scene_object and import_asset return the new ID in the result. Parse it to address the object in follow-up actions without scanning state.

• For read-only monitoring, disable all capabilities except reads. Untick all four feature checkboxes. State reads (the state, actions, and UI-elements endpoints) are never gated and still work.

• Use the sequencer state to verify playback. Each sequence entry includes its transport state (stopped, playing, or paused), position, duration, and loop flag — everything you need to confirm playback without looking at the screen.

Related

Integrate with Backstage via the REST API — integrate with Backstage via the general REST API (NDI routing, web pages, show-controller HTTP integration).

How to Control Backstage via the JSON Remote Control Protocol — control steppers, sequences, and broadcast schedulers over a JSON line protocol (TCP/UDP).

Settings Window Reference — full Settings window reference, including Remote Control options.

How to Use the AI Assistant — how to use the in-app AI Assistant (the Automation API's in-process sibling).