Runbook: Workflow Engine

Runbook: Workflow Engine

Last updated: 2026-03-30 Service owner: ODS Platform Team Pipeline status: STAGING_VERIFIED

1. Service Overview

Workflow Engine provides workflow definition, execution, step management, and transition logic for the ODS Platform. It supports multi-tenant workflow templates, instance lifecycle management, and publishes state-change events to Redpanda/Kafka.

Property Value
Language Go 1.24
Container port 8083
DB schema workflow on ods database
Event bus Kafka/Redpanda (redpanda:9092)
Coolify app UUID lcwgkwo8c8c8swwkkwgg00g8

2. Health Check

Endpoint: GET /health Expected response: HTTP 200

curl -sf https://workflow-engine.staging.orbusdigital.com/health

3. Staging URL

https://workflow-engine.staging.orbusdigital.com

4. Environment Variables

Variable Required Description
DATABASE_URL Yes postgres://ods:***@ods-postgres:5432/ods?search_path=workflow
JWT_SECRET Yes JWT secret for token verification
HTTP_PORT No Listen port (default 8083)
KAFKA_BROKERS No Kafka/Redpanda broker (default localhost:9092)
LOG_LEVEL No Log level (default info)
SERVICE_NAME No workflow-engine

5. How to Deploy

Via Coolify API

source ~/.env.adlc 2>/dev/null
curl -sf -X POST "https://app.coolify.io/api/v1/applications/lcwgkwo8c8c8swwkkwgg00g8/restart" \
  -H "Authorization: Bearer $COOLIFY_TOKEN"

Git-based deploy

Push to the staging branch of jniox/workflow-engine. Coolify source: app-jniox-ods (source_id: 2).

6. How to Check Logs

# On srv-staging (35.195.54.220):
docker logs --tail 200 -f $(docker ps -qf "label=coolify.applicationId=lcwgkwo8c8c8swwkkwgg00g8")

7. Common Issues and Fixes

Issue: Workflow instance stuck in transition

Symptom: Workflow instance stays in “transitioning” state.

Fix: Check logs for panic or timeout. Verify Kafka connectivity:

docker exec <container> wget -q -O- http://redpanda:9644/v1/status/ready

Issue: Go binary crash on startup

Symptom: Container restarts repeatedly, logs show “panic” or “fatal”.

Fix: Common causes: 1. DATABASE_URL missing or malformed 2. PostgreSQL workflow schema not migrated 3. JWT_SECRET not set

Verify schema:

psql "postgres://ods:ods-dev-2026@127.0.0.1:5433/ods" -c "SET search_path TO workflow; \dt"

Issue: Security review findings (7/10)

Context: Security review noted 3 findings. Check ~/dev/ops/reviews/workflow-engine/ for details.

8. How to Restart

docker restart $(docker ps -qf "label=coolify.applicationId=lcwgkwo8c8c8swwkkwgg00g8")

9. Dependencies

Dependency Type Details
PostgreSQL 17 Database ods-postgres:5432, schema workflow
Redpanda Event bus redpanda:9092
OID Auth JWT secret or JWKS for token verification

10. Monitoring / Alerting