Clustering

FlatRun supports clustering multiple agents together, giving you a unified view of deployments and system health across all your servers from a single dashboard.

How It Works

Clustering uses an invitation-based model. One node generates an invite token, another node accepts it, and they exchange API keys for secure bidirectional communication.

Once connected, each node periodically checks its peers. The UI and API aggregate deployments, statistics, and permitted management actions across the cluster.

Set up Fleet from the UI

1. Open Fleet and select Set up Fleet.
2. Confirm the server name and reachable agent URL suggested from Server Info.
3. Select Enable Fleet.
4. Use Invite server or Join Fleet to connect another server.

Existing deployments remain on their current servers. The connection adds management access only.

Orchestration and Routing

Choose one supported provider pair from the Cluster page:

K3s autoscaling reads the Kubernetes Metrics API. Install Metrics Server before activating a K3s workload. Swarm reads Docker statistics and refuses to scale when it cannot observe every running replica.

Scale-ready Deployments

A deployment must identify the service that can safely run as interchangeable replicas:

scaling:
  service: app
  stateless: true
  storage:
    mode: none

FlatRun checks the Compose definition before activation. Local volumes, privileged containers, host networking, external environment files, and services without an image block activation because they cannot be moved safely.

Activation creates the minimum replica count and waits for every replica to become ready. It then publishes the managed route and stops the original Compose service. A failed cutover restores the Compose route and removes the incomplete workload.

Lending Capacity

1. Connect both agents as peers.
2. Open the peer permissions on the server that may lend resources.
3. Enable Offer capacity and set CPU, memory, and replica limits.
4. Enable Use permitted Fleet capacity in the deployment autoscaling policy.

Swarm placement uses a peer-specific node label. A workload stays on its local node when no connected peer grants capacity. It cannot use a node that only grants read or deployment management access.

Every lending server must already belong to the same Docker Swarm. FlatRun verifies the Swarm identity and stops activation when a permitted peer belongs to a different Swarm.

Borrowed capacity requires CPU and memory limits under the service's Compose deploy.resources.limits. FlatRun rejects a peer when either per-replica limit exceeds that peer's grant.

CLI

# Check whether the deployment can be managed
flatrun deployments autoscale-compatibility shop

# Activate the saved workload and policy
flatrun deployments autoscale-activate shop

# Inspect the current policy and managed state
flatrun deployments autoscale shop

Configuration

The UI saves this configuration for you. For a manual setup, add the cluster section to each agent config:

# /etc/flatrun/config.yml
cluster:
  enabled: true
  server_name: "node-1"
  advertise_url: "https://node-1.example.com:8090"
  health_interval: "30s"
  request_timeout: "10s"

Joining Nodes

Clustering is managed entirely via the API (or the UI). Here's the flow:

1. Generate an Invite

On the node you want others to join:

curl -X POST http://localhost:8090/api/cluster/invite \
  -H "Authorization: Bearer $TOKEN"

This returns a one-time invite token that expires in 1 hour.

2. Accept the Invite

On the joining node:

curl -X POST http://localhost:8090/api/cluster/accept \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "invite_token": "THE_INVITE_TOKEN",
    "peer_url": "https://node-1.example.com:8090",
    "callback_url": "https://node-2.example.com:8090"
  }'

Both nodes exchange API keys and begin monitoring each other.

3. Verify the Connection

# Check cluster status
curl http://localhost:8090/api/cluster/status \
  -H "Authorization: Bearer $TOKEN"

# List all peers
curl http://localhost:8090/api/cluster/peers \
  -H "Authorization: Bearer $TOKEN"

Cluster API

Aggregated Responses

Cluster endpoints like /api/cluster/deployments and /api/cluster/stats return data grouped by node:

{
  "servers": {
    "node-1": {
      "name": "node-1",
      "online": true,
      "data": [... deployments ...]
    },
    "node-2": {
      "name": "node-2",
      "online": true,
      "data": [... deployments ...]
    }
  }
}

If a peer is offline, it appears with "online": false and an error message.

Security

• Encrypted Keys Peer API keys are encrypted at rest with AES-256-GCM
• One-Time Invites Invite tokens are single-use and expire after 1 hour
• Hashed Storage Only token hashes are stored in the database
• Independent Keys Each peer-to-peer connection uses independent API keys

Existing connections after an update

Agent startup runs schema migrations and an idempotent Fleet repair. Stored peer URLs and encrypted keys remain in place. Missing peer policies receive the default grants, and older peer credentials are restricted to their saved policy. Peers do not need to reconnect.

Removing a Peer

curl -X DELETE http://localhost:8090/api/cluster/peers/node-2 \
  -H "Authorization: Bearer $TOKEN"

Requirements

• Each node must be reachable by its peers at the advertise_url
• Port 8090 (or your configured API port) must be open between nodes
• Each node needs a unique server_name
• All nodes should run compatible agent versions