E2E Test Report -- Workflow Engine

E2E Test Report – Workflow Engine

Report ID: TEST-20260409-WF Date: 2026-04-09 Service: workflow-engine (ods-platform) Operator: ADLC Pipeline (automated) Session: Routine validation – Workflow Engine STAGING_RECOVERED since 2026-04-06

1. Executive Summary

Metric Value
Total tests executed 17
Passed 16
Failed 0
Partial 1
Pass rate 94%
Bugs found 0
Service status STAGING_RECOVERED since 2026-04-06
Verdict E2E_PASS

Workflow Engine demonstrates strong stability after its staging recovery. 16 of 17 E2E tests pass fully, covering the complete workflow lifecycle: definition CRUD, activate/deactivate/archive, instance creation/listing/cancellation, step completion with pass/fail results, and cross-tenant isolation. One test received a PARTIAL verdict (see Section 4). No bugs were found. The service was briefly returning 503 before 2026-04-06 and has since recovered to stable 200 responses.

Critical Findings

2. Test Environment

Parameter Value
Staging URL https://workflow-engine.staging.orbusdigital.com
Stack Rust / Actix-web / PostgreSQL 17 (schema: workflow)
Auth method HS256 JWT with tenant_id claim (via JWT_SECRET)
Auth provider Self-signed HS256 tokens (not OID RS256)
Database PostgreSQL 17, schema workflow
JWT algorithm HS256 (HMAC + SHA-256)
Previous session 2026-03-25 – 34/34 E2E pass
BA review 14/15 criteria met
Architect review PASS

3. Test Results

ID Test Name Category HTTP Verdict Notes
TC-WF-001 Health Check health 200 PASS /health returns 200, service operational
TC-WF-002 Ready Check health 200 PASS /ready returns 200, DB connected
TC-WF-003 Create Definition happy-path 201 PASS New workflow definition created with steps
TC-WF-004 List Definitions happy-path 200 PASS Returns definitions for tenant
TC-WF-005 Get Definition happy-path 200 PASS Single definition retrieved by ID
TC-WF-006 Update Definition happy-path 200 PASS Definition name/steps updated
TC-WF-007 Activate Definition (draft->active) happy-path 204 PASS State transition draft to active
TC-WF-008 Create Instance happy-path 201 PASS Instance created from active definition
TC-WF-009 List Instances happy-path 200 PASS Instances listed with ?definition_id filter
TC-WF-010 Get Instance happy-path 200 PASS Single instance retrieved with steps
TC-WF-011 Complete Step (result=pass) happy-path 200 PASS Step marked complete with pass result
TC-WF-012 Complete Step (result=fail) happy-path 200 PASS Step marked complete with fail result
TC-WF-013 Cancel Instance happy-path 204 PASS Running instance cancelled via PATCH
TC-WF-014 Deactivate Definition happy-path 204 PASS Active definition moved to inactive/archived
TC-WF-015 Delete Definition (409 with instances) validation 409 PASS Cannot delete definition that has instances – returns 409 Conflict as expected
TC-WF-016 Cross-Tenant Isolation multi-tenancy 404 PASS Tenant A cannot see Tenant B definitions or instances
TC-WF-017 Auth with HS256 Token auth 200 PARTIAL Authentication works with HS256 JWT but diverges from platform RS256 standard; functional but not aligned with OID

4. Partial Test Details

TC-WF-017: Auth with HS256 Token – PARTIAL

Behavior: The workflow engine authenticates requests using HS256 JWT tokens signed with a shared JWT_SECRET environment variable. Authentication itself works correctly – valid tokens are accepted, invalid/expired/missing tokens are rejected with 401.

Why PARTIAL: The ODS platform standard is RS256 JWT tokens issued by OID, validated via the OID JWKS endpoint. All other services (OID, DocStore, PDF Engine) use RS256. Workflow Engine’s use of HS256 means:

  1. Tokens are not issued by OID and cannot be verified against OID’s public keys
  2. The shared secret (JWT_SECRET) must be distributed to every service that needs to call Workflow Engine
  3. Token rotation requires coordinated secret rotation across all consumers
  4. No centralized token revocation via OID

Impact: Functional for current staging validation, but blocks production deployment until migrated to RS256/OID JWKS.

5. Workflow Lifecycle Coverage

Flow / Feature Test IDs Status
Definition CRUD (create, read, update, list) TC-WF-003 to TC-WF-006 Covered
Definition state machine (draft -> active -> archived) TC-WF-007, TC-WF-014 Covered
Instance lifecycle (create, list, get) TC-WF-008 to TC-WF-010 Covered
Step completion (pass and fail results) TC-WF-011, TC-WF-012 Covered
Instance cancellation TC-WF-013 Covered
Referential integrity (delete blocked by instances) TC-WF-015 Covered
Multi-tenant isolation TC-WF-016 Covered
Health and readiness probes TC-WF-001, TC-WF-002 Covered
JWT authentication (HS256) TC-WF-017 Partial

Flows Not Yet Covered by E2E

Flow Priority Notes
RS256 / OID JWKS authentication HIGH Must migrate from HS256 before production
Webhook/callback on step completion MEDIUM Spec may require event emission on state change
Redpanda event emission (CloudEvents) MEDIUM Platform standard: every state change emits to Redpanda
Pagination (definitions and instances) LOW Previously covered in 34-test suite (2026-03-25); not retested in this reduced set
Validation edge cases (empty name, nonexistent IDs) LOW Previously covered in 34-test suite (2026-03-25); not retested in this reduced set

6. Comparison with Previous Sessions

Metric 2026-03-25 (initial) 2026-04-09 (current) Delta
E2E tests 34 17 Reduced scope (focused lifecycle)
Pass rate 100% (34/34) 94% (16/17, 1 partial) -6% (auth divergence noted)
Bugs found 0 0 Clean
Staging status Stable RECOVERED (was 503 before 04-06) Recovered
BA review 14/15 14/15 Stable
Architect review PASS PASS Stable

The reduced test count (17 vs 34) reflects a focused lifecycle test rather than the exhaustive suite. The previously-covered categories (pagination, validation edge cases, detailed multi-tenancy) all passed in the 2026-03-25 run and were not re-executed. The 1 partial is a newly-flagged architectural concern, not a regression.

7. Recommendations

  1. Migrate to RS256 / OID JWKS (HIGH): This is the most important action item. Workflow Engine must validate tokens against OID’s JWKS endpoint (https://oid.staging.orbusdigital.com/.well-known/jwks.json) using RS256 instead of a shared HS256 secret. This aligns with the platform standard and is required for production deployment.

  2. Add Redpanda Event Emission: The ODS platform requires every state change to emit a CloudEvent to Redpanda. Workflow Engine should emit events for: definition created/activated/archived, instance created/completed/cancelled, step completed. This enables downstream consumers (ClickHouse analytics, Notification Hub).

  3. Re-run Full 34-Test Suite: The current session used a reduced 17-test set. Run the full suite (including pagination, validation, and detailed multi-tenancy) to confirm no regressions from the 503 recovery period.

  4. Document ?definition_id Requirement: The GET /api/instances endpoint requires ?definition_id as a query parameter. This should be documented in the API reference to avoid confusion for consumers expecting a global list.

  5. Production Readiness: Workflow Engine is functionally sound but blocked from production by the HS256 auth divergence. After RS256 migration and a full re-test, it will be ready for production promotion.

8. Verdict

E2E_PASS – Workflow Engine passes 16 of 17 end-to-end tests (94% pass rate) with 1 partial due to HS256 auth divergence from the platform RS256 standard. The core workflow lifecycle is fully operational: definitions CRUD, state machine transitions, instance management, step completion, cancellation, and tenant isolation all work correctly. Zero bugs found. The service has recovered from its brief 503 period and is stable on staging. Migration to OID RS256 authentication is the primary action item before production promotion.

Generated by Documentarian Agent on 2026-04-09.