How to Use the AI Assistant

How to Use the AI Assistant

Backstage includes a built-in AI Assistant that connects to large language models (LLMs) for conversational help and local image generation. Configure an LLM provider, then chat with the assistant or generate images directly inside Backstage.

[USER-ACTION-REQUIRED: Attach a screenshot of the AI Assistant window (Chat tab visible) here]

Before you start

• An LLM provider is reachable from this machine (local Ollama instance, or a cloud API key for OpenAI / Anthropic / Mistral / Ollama Cloud)
• For image generation: a Stable Diffusion model file (.safetensors or .ckpt) available on local storage, or internet access to download one from HuggingFace

Steps

1. Configure an LLM provider

Open Settings and expand the AI section. Backstage organises providers into two groups:

• Local — Ollama (runs on the same machine or local network). Click Localhost to auto-fill the default URL http://localhost:11434. No API key required.

• Cloud — Ollama Cloud, OpenAI, Anthropic, or Mistral. Enter the Server URL and API Key for your provider. The connection status indicator turns green when Backstage can reach the endpoint.

Click Refresh to fetch the list of available models from the provider.

2. Assign providers to tasks

Below the provider list, the Tasks section lets you choose which provider and model to use for each AI task:

• Chat — the conversational assistant in the Chat tab.

• Vision — screenshot analysis, used by the Training features.

• Orchestration — planning/reasoning for advanced Training features. Falls back to the Vision provider if not configured.

• Image Generation — used by the Generate tab (Stable Diffusion runs locally; this task assignment is for future LLM-based generation).

For each task, select a Provider from the dropdown (only connected providers are selectable), then pick a Model.

[USER-ACTION-REQUIRED: Attach a screenshot of the Settings window with the AI section expanded here]

3. Open the AI Assistant window

Go to About in the menu bar and select Ai Assistant. The window opens as a floating panel.

4. Chat with the assistant

The Chat tab is the default view. A status bar at the top shows the active provider, model name, and connection state (green = connected, red = disconnected).

Type your question in the input field at the bottom and press Enter or click Send. The assistant streams its response in real time. Use Cancel to stop a response mid-stream.

The Mode dropdown switches the assistant between:

• Chat — plain text conversation (default).

• User UI — the assistant can see your current Backstage UI via screenshots (requires a Vision-capable model).

• Own UI — the assistant operates on a dedicated Backstage window (requires a Vision-capable model).

Additional controls:

• New Chat — clears the conversation history and starts fresh.

• Copy Conversation — copies the entire chat to the clipboard.

• Copy (per message) — copies a single AI response to the clipboard.

• Auto-scroll — keeps the chat scrolled to the latest message.

5. Generate images (Generate tab)

The Generate tab uses Stable Diffusion to create images locally on the server's GPU.

First, configure a model in Settings > AI > Image Generation. Backstage scans the model directory (default: Models\StableDiffusion next to the Backstage executable) for .safetensors or .ckpt files. You can also search and download models directly from HuggingFace in the Settings panel.

In the Generate tab, type a prompt and press Enter. The model loads on first use (this may take a moment) and unloads automatically after 2 minutes of inactivity to free GPU memory. A progress bar shows the current step. Click Import as Asset to bring the generated image into your project.

Options explained

Settings > AI — Provider configuration

Generate tab — Image generation settings

Common Mistakes

• Status shows "Disconnected" after entering an API key — the key is saved only after you leave the input field (click elsewhere or press Tab). If the status dot stays red, verify the key is correct and the Server URL matches the provider's API endpoint. Click Refresh to retry.

• Send button is greyed out — the Chat task has no connected provider assigned. Check Settings > AI > Chat and ensure both a Provider and a Model are selected, and the provider status shows "Connected".

• "No model selected" in the Generate tab — go to Settings > AI > Image Generation and select a Stable Diffusion model from the dropdown. If no models appear, place a .safetensors file in the model directory, or use the HuggingFace download section in Settings to fetch one.

• Image generation fails with out-of-memory — Stable Diffusion runs on the server GPU alongside Backstage rendering. Reduce the image Width/Height or close unused outputs to free GPU memory.

Tips

• Backstage polls each provider every 5 seconds. The connection indicator updates automatically — you do not need to click Refresh to check status.

• The Stable Diffusion model unloads from GPU memory automatically after 2 minutes of inactivity. This prevents it from competing with production playback.

• You can use different providers for different tasks. For example, a fast local Ollama model for Chat and a cloud provider for Vision.

• Use Ctrl+Enter for a new line in the Generate prompt. Enter alone submits the prompt.

Related

Settings Window Reference — Settings Window Reference, including all global configuration categories.

Your First Project — Your First Project, for getting started with Backstage basics.