Orchestrator

The orchestrator is EnderCloud's central control-plane process. It is the only component that chooses hosts, changes lifecycle state, forms sessions, writes transfer commands, or interprets group policy. Run one active orchestrator against a PostgreSQL database.

The entry point is orchestrator/src/index.ts.

Responsibilities

The orchestrator:

• migrates the database and synchronizes group and template configuration;
• maintains group capacity and warm pools;
• forms minigame sessions from atomic parties;
• reserves CPU and memory on one eligible execution host;
• records durable create commands and dispatches them to agents;
• processes Paper readiness, presence, assignment, and game lifecycle events;
• persists and retries Velocity transfer commands;
• renews hubs, drains instances, and coordinates host maintenance;
• reconciles desired state with agent container inventories;
• aggregates monitoring samples and tracks operational incidents;
• exposes private HTTP and OpenAPI endpoints for agents, plugins, and the dashboard.

It does not mount a Docker socket, read remote host files, allocate game ports, or connect players to Minecraft servers. Those jobs belong to agents and Velocity.

Startup sequence

The process performs these steps before readiness returns HTTP 200:

1. Parse and validate every orchestrator environment variable.
2. Create the configured group and template directories if they are missing.
3. Apply pending SQL migrations.
4. Reject an unsafe multi-host migration if an active legacy instance has no host_id.
5. Parse, validate, checksum, and synchronize groups and template layers.
6. Connect the Redis event bus.
7. Recover interrupted instance-start commands.
8. Reconcile variant startup policy, host inventory, capacity, transfers, and incidents once.
9. Start the API and periodic control loops.

GET /health/live reports that the process can serve HTTP. GET /health/ready reports that the startup sequence completed. Compose depends on readiness, not only liveness.

The orchestrator reads group and template files only during startup. Restart it after editing YAML or template content.

Run it

The normal deployment starts the whole stack from the repository root:

Copy-Item .env.example .env
docker compose up --build

For a development process, first make PostgreSQL and Redis reachable, then run:

cd orchestrator
bun install --frozen-lockfile
bun run dev

bun run dev watches TypeScript files. bun run start starts the same entry point without watch mode.

Control loops

Each named loop refuses to overlap with its previous run. A slow tick is skipped instead of running the same controller twice.

HTTP API

The generated OpenAPI page is available at /openapi. Treat it as the request and response schema reference. The route groups are:

Every request receives an x-request-id. If the caller provides one, the orchestrator keeps it. Durable work also adds a command identifier to logs and agent calls.

The API has no authentication in the current MVP. Bind it to loopback or a private interface.

Environment variables

This table lists every variable read by the orchestrator process. Defaults come from orchestrator/src/config.ts, not from Compose.

Durations require an integer and one of ms, s, m, h, or d. A bare number is invalid. Paths are resolved against the process working directory.

The root compose.yml sets internal ports and configuration paths explicitly. It forwards the startup, scheduler, host, executor, and log settings shown in its orchestrator.environment section. Docker Compose only passes declared values into the container. If you tune an incident variable that is not declared there, add the same variable to that environment section or run the orchestrator outside that Compose service.

Compose-only variables

These values are consumed by Docker Compose or the PostgreSQL image. The orchestrator process does not read them directly.

ORCHESTRATOR_PUBLISH_ADDRESS does not change the URL used inside the Compose network. The primary agent uses http://orchestrator:8080. Remote agents need a private address that resolves to the published central host.

Database migrations

Startup applies migrations automatically. To apply them without starting the API:

cd orchestrator
$env:DATABASE_URL = "postgres://endercloud:change-me@localhost:5432/endercloud"
bun run db:migrate

Generate a migration after changing src/db/schema.ts:

bun run db:generate

Review generated SQL before committing it. Do not edit an already deployed migration.

Logging and diagnosis

The orchestrator emits one JSON object per line. Common context fields include service, component, requestId, commandId, task, runId, hostId, instanceId, groupId, and variantId. URL credentials and environment values whose names contain password, secret, token, key, or credential are redacted.

Use debug only during diagnosis because scheduler and HTTP completion events are noisy at that level. Startup failures appear through the bootstrap logger even when full configuration cannot load.

For lifecycle problems, correlate the instance detail in the dashboard with structured logs, the durable command, the host health state, and any active incident. For a blocked variant, inspect the retained log tail and runtime before requesting a retry.

See Environment configuration for removed names and Agent for the execution-side settings.