Fleet Management

Overview

Fleet Management is the central control plane for all distributed edge gateways. Instead of SSH-ing into each device individually, Fleet Management provides a single dashboard to register, monitor, and manage all devices remotely — from restarting a connector to rolling out a complete containerized application.

Every device registered here also receives its own certificate-based identity through the platform's PKI module, which it uses to authenticate itself afterwards. See Additional Services / PKI for details on how a device obtains, renews, and eventually loses that identity.

Key Features

Device Dashboard

The dashboard provides a real-time overview of all registered edge devices:

• KPI Cards — Total devices, online, warnings, offline, updating (auto-refreshing)
• Device Table — Sortable, filterable, paginated list of all devices with status, site, health score, and last heartbeat
• Device Detail Drawer — Click any device to view CPU/RAM/disk usage, running connectors, other containers on the host, and event history
• Configurable Auto-Refresh — Choose update interval (5s, 10s, 30s, 60s, or off)

Device Registration

1. Click Register Device in the dashboard
2. Enter a Device Name and Site (select existing or create a new one)
3. Registration automatically requests a device-specific enrollment token from the PKI module (short-lived, single-use) and returns it together with a ready-to-use install command

Run the install command on the target device to set up the agent automatically:

curl -sSL http://<management-host>/api/fleet/agent/install.sh | \
  ENROLLMENT_TOKEN=<token> \
  DEVICE_ID=<uuid> \
  MANAGEMENT_URL=http://<management-host> bash

The device appears in the dashboard within seconds. Behind the scenes, the agent uses the enrollment token exactly once to obtain its own certificate from the PKI module before it starts sending heartbeats — see Additional Services / PKI for the full onboarding sequence.

Monitoring

Each device sends a heartbeat at a regular interval (default: every 30 seconds) containing:

• System Metrics — CPU, RAM, and disk usage
• Device Information — OS, kernel, architecture, uptime
• Connector Status — List of all running ALP-CONNEX connectors with name, type, and status
• Additional Containers — Other Docker containers running on the device that are not ALP-CONNEX connectors, so operators can see what else occupies the host
• Capabilities — What the device offers: container runtime, bundle schema support, and detected hardware devices (see Capabilities below)
• Applications — State of every application bundle deployed to the device (see Edge Applications)

The heartbeat is also the channel through which the device receives its pending commands — the response to each heartbeat carries the queue.

Device Status

The status shown in the dashboard is reported by the agent, not calculated centrally:

Status	Meaning
ONLINE	A heartbeat arrived within the expected window
OFFLINE	No heartbeat for more than 2× the device's heartbeat interval — set by a background service in the management backend, and reset to ONLINE as soon as a heartbeat arrives again

Health score and the Warning/Updating states. Devices carry a healthScore field and the dashboard has KPI cards for Warnings and Updating. These are not derived from live metrics: the health score is set to 100 when a device is registered and is never recalculated, and no code path assigns WARNING or UPDATING to a real device — only the demo seed data uses them. Use the CPU/RAM/disk values in the device drawer to judge a device's health, not the health score.

Capabilities

On every heartbeat the agent reports what the host can actually do. The management backend uses this list to decide whether an application bundle may be deployed to a device, and the agent re-checks the identical list before it acts — so a bundle the UI considers compatible is never rejected on the device.

Capability	Reported when
artifact.atomic-install	Always
application-bundle	A container runtime is reachable; carries the supported bundle schema as an attribute
runtime.docker	A container runtime is reachable; the version is the Docker server version
secret-ref	A container runtime is reachable
registry-credential-ref	A container runtime is reachable
device.dma-heap	/dev/dma_heap/system exists
device.fastrpc-cdsp	/dev/fastrpc-cdsp exists
device.fastrpc-cdsp-secure	/dev/fastrpc-cdsp-secure exists

The three device.* capabilities are what make NPU-accelerated edge AI workloads possible — they are present on Qualcomm-based boards and absent on ordinary x86 gateways.

Remote Commands

Commands are queued from the dashboard and picked up by the device on its next heartbeat. The device reports the outcome, which is tracked in the device's event log.

Command	Description
Restart Connectors	Restarts all connector containers on the device
Connector Start / Stop / Restart	Acts on a single connector container
Update	Pulls a specified image and restarts the target container
Agent Update	Replaces the agent itself, then restarts it
Configure	Pushes a configuration payload to the device
Certificate Renew	Triggers an immediate certificate renewal (see Additional Services / PKI)
AI Chat	Runs a prompt against a model on the device and returns the answer (see Additional Services / AI Service)
Application Bundle Apply	Deploys or validates an application bundle revision (see Edge Applications)
Application Bundle Remove	Removes an application's containers, networks, and staged files from the device

Two command types are long-running and are therefore confirmed as RUNNING before execution starts, so a restart of the agent cannot lose them: AI Chat and Application Bundle Apply.

Site Management

Devices can be assigned to sites. Sites are created during device registration and allow filtering and grouping in the dashboard.

Device Groups

Devices can also be organized into device groups — a more flexible grouping than sites, used for filtering and bulk assignment. A device group can optionally be pinned to a specific certificate authority (CA): every device placed in that group then receives its certificates from that CA instead of the platform's default one. This is useful for separating trust between sites, customers, or environments. See Additional Services / PKI for how certificate authorities work.

Connector Templates

Reusable connector configurations that can be applied to device profiles for consistent, fleet-wide deployment instead of configuring each device by hand.

Edge Applications

Beyond connectors, Fleet Management can deploy arbitrary containerized workloads to edge devices — for example a local AI inference service, a protocol gateway, or a data pre-processor. These are called applications, and they are delivered as application bundles.

Concepts

Concept	Meaning
Application	A named workload, e.g. Edge AI – Genie. It is a container for revisions and has a stable application ID
Revision	An immutable, numbered version of the application's specification. Editing never changes a revision — it creates a new one
Spec	The JSON document describing what to run: services, volumes, networks, devices, health checks, and requirements
Spec digest	A SHA-256 fingerprint of the revision's spec. The device recomputes it and refuses the bundle if it does not match, which guarantees that what you approved is exactly what runs
Deployment	The record of one revision being rolled out to one device

The specification

A revision's spec is a JSON document with these top-level sections:

Section	Purpose
requirements	What the device must provide: platforms (OS/architecture), capabilities, and devices (paths that must exist)
registries	Private registries to authenticate against, each optionally referencing a registry credential
networks	Docker networks to create for this application
volumes	Named volumes, optionally bound to a host path via sourcePath
artifacts	Files to download and place into a volume before start, each pinned by SHA-256
services	The containers themselves: image, command, environment, ports, mounts, devices, networks, dependencies, restart policy, health check
rollout	Rollout timeout and, for a rollback, the revision to return to

Two rules are enforced and will reject a revision:

• Images must be pinned by digest (image@sha256:…). A mutable tag such as :latest is refused, because it would make the deployed result unreproducible.
• Service dependencies must be acyclic. The agent derives the start order from dependsOn; a cycle is rejected.

Deploying an application

1. Open Fleet Management → Applications and create an application, then a revision containing the spec.
2. Click Deploy and pick a target device.
3. Click Check Compatibility. The backend compares the revision's requirements against the capabilities from the device's most recent heartbeat and reports whether the device qualifies.
4. Confirm the deployment. An Application Bundle Apply command is queued and picked up on the next heartbeat.

On the device the rollout runs in two phases:

• Stage — resolve credentials, download and verify artifacts, pull all images. Nothing that is currently running is touched yet.
• Apply — create the networks, start the services in dependency order, and wait for each to become ready (respecting its health check). Only then is the revision marked active and the previous one superseded.

If the apply phase fails, the agent rolls back to the previous revision automatically and reports the failure. A rollout interrupted by a crash or reboot is reconciled on the next agent start.

A deployment can also be sent as a dry run, in which the device validates the bundle against its own capabilities and reports the result without changing anything.

The bundled example application

A fresh installation already contains one application, VENTUNO Edge AI – Genie (ventuno-edge-ai). It is seeded on backend startup and serves both as a working reference spec and as the inference back end for the edge chat. It deploys two services:

Service	Port	Purpose
genie	9001	Runs the language model on the Qualcomm NPU. Exposes an OpenAI-compatible API.
ai-service	8090	Translates between the agent's /api/ai/chat contract and the OpenAI-compatible API, and adds retrieval and tool calling.

ai-service declares dependsOn: genie with condition healthy, so the model is loaded before the adapter starts. Both are pinned by image digest.

Requires a credential before it can be deployed. The ai-service image is hosted in a private registry, and the spec points at the registry credential alpscale-registry. Create it first under Fleet Management → Credentials (see Application Credentials) with type Registry, reference alpscale-registry, and the server registry.alpscale.io, otherwise the rollout stops during the stage phase with CREDENTIAL_RESOLUTION_FAILED.

Because the agent reaches the adapter on 127.0.0.1:8090 — the default of its AI_SERVICE_URL setting — no further configuration is needed once the application is active. The seeder only ever adds what is missing, so revisions you create yourself are never modified or overwritten.

Rollback and removal

• Rollback — from the deployment history, return a device to a previously deployed revision.
• Remove — removes the application from one device: its containers, its networks, and its staged files under the agent's applications directory.
• Delete — removes the application and all its revisions from the platform. A removal command is queued automatically for every device that still has it deployed, so devices do not keep orphaned containers.

Troubleshooting a failed rollout

The device reports a machine-readable error code with every failure:

Code	Meaning
SPEC_DIGEST_MISMATCH	The spec received does not hash to the digest the backend announced — the payload was altered in transit
UNSUPPORTED_BUNDLE_SCHEMA	The device's agent does not understand this bundle schema version
CAPABILITY_MISSING / CAPABILITY_VERSION_UNSUPPORTED / CAPABILITY_ATTRIBUTE_MISMATCH	The device does not satisfy a requirement — check the heartbeat's capability list
PLATFORM_UNSUPPORTED	Wrong OS or CPU architecture
DEVICE_UNAVAILABLE	A required device path does not exist on the host
IMAGE_NOT_PINNED	A service image is not pinned by digest
DEPENDENCY_CYCLE	dependsOn forms a cycle
CREDENTIAL_RESOLUTION_FAILED	A referenced secret or registry credential could not be fetched
BUNDLE_STAGE_FAILED	Artifact download or image pull failed
BUNDLE_ROLLOUT_FAILED	A service did not become ready in time; the previous revision was restored

Application Credentials

Applications frequently need secrets — an API key for a service, or a login for a private container registry. These are stored centrally under Fleet Management → Credentials and are never written into a bundle spec.

Type	Contains	Referenced from the spec by
Secret	A single value	secretRef on a service environment variable
Registry	Username and password	credentialRef on a registry entry

A credential has a short reference (lower-case letters, digits, ., _, -) that the spec points at. The reference and the type are immutable once created — only the value can be rotated.

How the value reaches the device. It is not pushed with the bundle. When the agent processes an Application Bundle Apply command and the spec contains at least one reference, it calls back to the management API over its mutual-TLS connection and asks for exactly the references in that payload. The backend verifies that the command belongs to the calling device, that it really is a bundle command, and that it is currently in flight, before it decrypts and returns the values. Every resolution is written to the audit log.

On the device, secrets stay in memory and are injected as environment variables. Registry logins are written to a temporary Docker configuration file readable only by the agent and removed afterwards — no docker login and no credentials in the daemon's persistent configuration.

Encryption at rest. Values are encrypted with AES-GCM using a key derived from the ALPCONNEX_ENCRYPTION_KEY environment variable of the management backend. The backend refuses to start without it. Read endpoints never return a stored value.

Back up this key. Losing ALPCONNEX_ENCRYPTION_KEY makes every stored credential permanently unreadable; they would all have to be re-entered by hand.

Agent Installation

Prerequisites

• Docker must be installed on the target device
• Root access for installation
• Network connectivity to the management server

Two installation methods

The agent can run either as a container or as a native system service. Both are fully supported; pick one per device.

	Container (install.sh)	Native service (install-agent.sh)
Runs as	A Docker container named connex-agent, managed by a generated docker-compose.yml	A systemd unit connex-agent.service
Binary / image	Agent image pulled from the management server	Single static binary in /usr/local/bin/connex-agent
Install directory	/opt/connex-agent — holds agent.env, docker-compose.yml, and data/, which is mounted into the container	/var/lib/connex-agent, with the configuration in /etc/connex-agent
Inspect logs with	docker logs connex-agent	journalctl -u connex-agent.service
Best for	Hosts already managed through Docker	Devices where the agent must survive Docker restarts, or where it manages Docker itself

The install command shown in the UI uses the container method:

curl -sSL http://<management-host>/api/fleet/agent/install.sh | \
  ENROLLMENT_TOKEN=<token> \
  DEVICE_ID=<uuid> \
  MANAGEMENT_URL=http://<management-host> bash

Either way the script will:

1. Check prerequisites (Docker, architecture)
2. Download the agent for the device's architecture
3. Write the configuration and the device identity
4. Start the agent and enable it to start on boot

Agent Configuration

Setting	Required	Default	Description
MANAGEMENT_URL	Yes	—	Management server URL
DEVICE_ID	Yes	—	The device ID assigned by the backend at registration. The agent never generates one itself and will not start without it
ENROLLMENT_TOKEN	First start only	—	One-time PKI enrollment token from the UI, used to obtain the device's first certificate
HEARTBEAT_INTERVAL	No	30	Seconds between heartbeats (10–300)
CERT_RENEW_BEFORE	No	7d	How long before certificate expiry the agent renews it automatically
AGENT_MTLS_CA_PATH	No	—	PEM CA bundle used to trust the management API's TLS certificate
APPLICATIONS_PATH	No	/var/lib/connex-agent/applications	Where application bundle revisions are staged and their state is kept
AI_SERVICE_URL	No	http://127.0.0.1:8090	Local inference endpoint used to answer AI Chat commands
STACK_VERSION	No	dev	Reported software version

Persistent Data

The agent stores its data locally, including:

File	Purpose
device-id	Unique device identifier
device-key.pem	The device's private key (generated locally, never leaves the device)
device-cert.pem	The device's current certificate, issued by the platform's PKI
device-chain.pem	The certificate chain needed to validate the device certificate
device-config.json	Configuration pushed from the server
applications/	Staged application bundle revisions and the record of which one is active per application

The enrollment token itself is not stored — it is used once during onboarding and discarded. See Additional Services / PKI for how these files are obtained and kept up to date.

Troubleshooting

Device not appearing in dashboard

1. Check the agent logs — docker logs connex-agent for a container install, journalctl -u connex-agent.service for a native install
2. Management URL reachable? curl http://<host>/api/fleet/devices/stats
3. Enrollment token expired? These tokens are short-lived by design — register the device again (or reissue a token from Certificate Management) to get a fresh one
4. Was DEVICE_ID supplied at install time? Without it the agent exits at startup

Device shows "Offline"

A device is marked offline once no heartbeat has arrived for more than twice its heartbeat interval:

• Is the agent running? docker ps -f name=connex-agent or systemctl status connex-agent.service
• Check network connectivity to the management server

No connectors reported

The agent discovers connectors by inspecting running Docker containers. Requirements:

• Docker socket must be mounted
• Connector containers must use the ALP-CONNEX naming scheme

Application deployment is rejected as incompatible

Compare the revision's requirements with the capability list in the device's latest heartbeat — it is shown in the device drawer. A device only advertises application-bundle and runtime.docker when a container runtime is actually reachable, and the device.* capabilities only when the corresponding device node exists on the host.