Skip to content

Command-Line Interface (CLI) Reference

The orchestration-engine package provides a standalone CLI tool for validating, executing, resuming, and inspecting workflow runs.

Entry Points

The CLI can be invoked in two equivalent ways:

# As an installed console script
orchestration-engine <command> [options]

# As an executable Python module
python -m orchestration_engine <command> [options]

Security & Runner Trust Model

The run and resume subcommands accept a --runner module:function flag.

[!WARNING] Specifying --runner dynamically imports Python code via importlib.import_module and executes the specified callable. To resolve local modules, the current working directory (os.getcwd()) is prepended to sys.path.

Running a workflow with --runner carries the same trust level as executing a Python script. Only pass runner paths from trusted sources.

Exit Codes

All CLI commands follow strict, machine-parseable exit code contracts:

Exit Code Meaning Description
0 Success Workflow execution succeeded, or query/validation command succeeded.
1 Workflow Failure Workflow ran to completion but finished with status failed or cancelled.
2 Usage / Definition Error Invalid CLI arguments, schema validation failure, file not found, or run ID not found.
3 Internal / Database Error Database connection/query failure (PersistenceError) or unhandled runtime error.

Subcommands

1. validate

Validates a workflow definition file against the formal workflow schema. Supports JSON files, or YAML files if the optional yaml extra is installed (pip install 'orchestration-engine[yaml]').

orchestration-engine validate <workflow_file> [--json]

JSON Schema (--json): 

{
  "workflow_id": "wf-example-1",
  "tasks_count": 2,
  "valid": true
}

2. run

Executes a workflow to completion from a definition file.

orchestration-engine run <workflow_file> \
    --runner <module:function> \
    [--db <path>] \
    [--run-id <id>] \
    [--max-concurrency <N>] \
    [--executor <thread|process>] \
    [--json]
  • --runner: (Required) Path to the task runner callable in module:function syntax.
  • --db: Path to an SQLite database file. If omitted, an in-memory database is used.
  • --run-id: Optional custom identifier for the run. If omitted, a UUID is generated.
  • --max-concurrency: Maximum concurrent tasks (default: 1).
  • --executor: Concurrency backend: thread (default) or process.
  • --json: Output machine-readable JSON summary.

JSON Schema (--json): 

{
  "run_id": "477fdb0e0b774caf85cf4b6f80c47432",
  "workflow_id": "wf-example-1",
  "status": "succeeded"
}

3. resume

Resumes an interrupted or partially-failed workflow run from its persisted database state.

orchestration-engine resume <run_id> \
    --runner <module:function> \
    --db <path> \
    [--max-concurrency <N>] \
    [--executor <thread|process>] \
    [--json]
  • --db: (Required) Path to the SQLite database containing the persisted run.

JSON Schema (--json): 

{
  "run_id": "477fdb0e0b774caf85cf4b6f80c47432",
  "status": "succeeded"
}

4. status

Queries the current status of a workflow run from the database.

orchestration-engine status <run_id> --db <path> [--json]

JSON Schema (--json): 

{
  "run_id": "477fdb0e0b774caf85cf4b6f80c47432",
  "status": "succeeded"
}

5. history

Retrieves the complete state transition audit log for a run, ordered chronologically.

orchestration-engine history <run_id> --db <path> [--json]

JSON Schema (--json): 

[
  {
    "id": 1,
    "workflow_id": "wf-example-1",
    "task_id": "fetch-data",
    "old_state": "pending",
    "new_state": "ready",
    "timestamp": 1792930540.132,
    "attempt_count": 0
  },
  {
    "id": 2,
    "workflow_id": "wf-example-1",
    "task_id": "fetch-data",
    "old_state": "ready",
    "new_state": "running",
    "timestamp": 1792930540.133,
    "attempt_count": 1
  }
]

6. runs

Lists workflow runs stored in the database, ordered newest first.

orchestration-engine runs --db <path> [--workflow <workflow_id>] [--json]
  • --workflow: Optional filter to list runs matching a specific workflow ID.

JSON Schema (--json): 

[
  {
    "run_id": "477fdb0e0b774caf85cf4b6f80c47432",
    "workflow_id": "wf-example-1",
    "started_at": 1792930540.132,
    "finished_at": 1792930540.134,
    "status": "succeeded"
  }
]

7. plan

Analyzes workflow structure and computes topological execution stages, critical path, and maximum parallelism without executing side effects.

orchestration-engine plan <file> [--json]
  • <file>: (Required) Path to the workflow JSON or YAML file.
  • --json: Format the execution plan output as structured JSON.

Example: 

orchestration-engine plan workflow.json

Output: 

Workflow: etl-pipeline
Total Tasks: 8
Max Parallelism: 4 concurrent workers
Critical Path: extract -> chunk -> embed -> index

Execution Stages:
  Stage 0: extract
  Stage 1: chunk_a, chunk_b, chunk_c, chunk_d
  Stage 2: embed
  Stage 3: index

8. dlq

Inspects and manages the persistent Dead Letter Queue (DLQ) for failed tasks.

orchestration-engine dlq <list|purge> [--db <path>] [--workflow <id>] [--limit <n>] [--older-than <sec>] [--json]
  • list: View captured task failure messages, stack traces, and input payloads.
  • purge: Delete captured failure messages matching filters.
  • --db: Path to the SQLite database (defaults to orchestration.db).
  • --workflow: Filter messages by workflow identifier.
  • --limit: Maximum number of messages to list (default: 50).
  • --older-than: Purge messages older than $N$ seconds.
  • --json: Output as structured JSON.

9. trace

Exports run execution spans into Google Chrome Trace Event Format (compatible with chrome://tracing and Perfetto UI).

orchestration-engine trace <run_id> --db <path> [-o trace.json]
  • run_id: Workflow run identifier.
  • --db: Path to the SQLite database.
  • -o, --output: Optional output JSON file path (defaults to stdout).

10. signal

Delivers a human-in-the-loop approval or external webhook payload to a workflow run.

orchestration-engine signal <run_id> <signal_name> [--payload <json>] [--db <path>]
  • run_id: Workflow run identifier.
  • signal_name: Identifier of the signal (e.g. approval, webhook, review).
  • --payload: JSON payload string (defaults to {}).
  • --db: Path to the SQLite database.

11. coordinator

Launches the zero-dependency distributed HTTP coordinator server. Orchestrates remote task polling, heartbeats, and run completion reporting across multiple worker agents over HTTP.

orchestration-engine coordinator [--host <host>] [--port <port>] [--db <path>]
  • --host: Host address to bind (default: 0.0.0.0).
  • --port: Port number to listen on (default: 9100).
  • --db: Path to SQLite database (defaults to orchestration.db).

12. worker

Launches an autonomous distributed worker daemon that connects to a remote coordinator over HTTP, polls available tasks, executes them, and streams back heartbeat status and completion payloads.

orchestration-engine worker --coordinator <coordinator_url>
  • --coordinator: (Required) Coordinator HTTP endpoint (e.g. http://127.0.0.1:9100).

13. supervisor

Scans active workflow runs in SQLite for deadlocked, abandoned, or zombie tasks (stuck in running past a timeout) and applies automated self-healing remedies.

orchestration-engine supervisor [--db <path>] [--timeout <seconds>] [--remedy <RETRY|ABORT_RUN|NOTIFY>] [--json]
  • --db: Path to the SQLite database (defaults to orchestration.db).
  • --timeout: Stalled task duration threshold in seconds (default: 300.0).
  • --remedy: Automated healing action: RETRY (default, resets task to ready), ABORT_RUN (marks run failed), or NOTIFY (records incident only).
  • --json: Output recorded incident details as structured JSON.

14. replicate

Manages Change Data Capture (CDC) streaming between primary and read-replica SQLite databases over stdlib HTTP without external brokers.

# On primary node: Start HTTP CDC server
orchestration-engine replicate --serve [--host 0.0.0.0] [--port 9200] [--db primary.db]

# On replica node: Pull incremental batch from primary
orchestration-engine replicate --sync --primary http://10.0.0.1:9200 [--db replica.db] [--json]
  • --serve: Runs the primary CDC replication HTTP server daemon.
  • --sync: Executes a single-pass synchronization against a primary CDC endpoint.
  • --primary: URL of the primary CDC server (required for --sync).
  • --host: Host to bind (default: 0.0.0.0).
  • --port: Port to listen on (default: 9200).
  • --db: SQLite database file path.

15. compact

Prunes fine-grained event logs for terminal runs, exports compressed .tar.gz cold archives, and reclaims disk space via database vacuuming.

orchestration-engine compact [--db <path>] [--compact] [--older-than-days <N>] [--archive-tar <path>] [--vacuum] [--json]
  • --db: Path to the SQLite database.
  • --compact: Prune fine-grained event log entries for completed runs.
  • --older-than-days: Filter runs finished older than N days (default: 30).
  • --archive-tar: Export eligible completed runs into a compressed .tar.gz archive.
  • --delete-archived: Remove exported runs from the SQLite database.
  • --vacuum: Execute non-blocking SQLite database vacuum to reclaim disk space.
  • --json: Output compaction summary as structured JSON.

Observability & Logging

By default, the orchestration_engine library attaches a NullHandler to avoid hijacking parent application logs.

When running via the CLI, use the --log-level flag to enable structured standard logging:

orchestration-engine --log-level INFO run examples/simple_workflow.json --runner examples.runners:run

Supported log levels: DEBUG, INFO, WARNING, ERROR.

Structured log events emit structured extra fields: - workflow_id: Workflow identifier - run_id: Run identifier - task_id: Task identifier - old_state: Previous task state - new_state: Next task state - attempt: Current attempt counter