Runbook: Notification Hub

Runbook: Notification Hub

Last updated: 2026-03-30 Service owner: ODS Platform Team Pipeline status: STAGING_VERIFIED

1. Service Overview

Notification Hub manages multi-channel notifications (email, in-app, webhook) for the ODS Platform. It consumes events from Redpanda, stores notification state in PostgreSQL, and delivers emails via Resend. Supports RS256 JWT verification from OID.

Property Value
Language Rust (Actix-web)
Container port 8080
DB schema notifications on ods database
Event bus redpanda:9092
Email provider Resend (resend.com)
Coolify app UUID jwgokkggkosokss4ok4wss80

2. Health Check

Endpoint: GET /health Expected response: HTTP 200

curl -sf https://notification-hub.staging.orbusdigital.com/health

3. Staging URL

https://notification-hub.staging.orbusdigital.com

4. Environment Variables

Variable Required Description
DATABASE_URL Yes postgres://ods:***@ods-postgres:5432/ods?search_path=notifications
SERVER_HOST No Bind address (default 0.0.0.0)
SERVER_PORT No Listen port (default 8085, mapped to 8080)
KAFKA_BROKERS No Redpanda broker address
JWT_SECRET Alt HS256 secret (dev)
OID_ISSUER Yes https://oid.staging.orbusdigital.com
OID_AUDIENCE No JWT audience claim
RESEND_API_KEY No Resend API key for email delivery (unset = emails stay Pending)
EMAIL_FROM No Sender address (default ODS Platform <noreply@orbusdigital.com>)
RUST_LOG No Log level
SERVICE_NAME No notification-hub
REDPANDA_BROKERS No redpanda:9092
REDPANDA_TOPIC No notifications.events
JWT_PUBLIC_KEY Prod RSA public key from OID JWKS

5. How to Deploy

Via Coolify API

source ~/.env.adlc 2>/dev/null
curl -sf -X POST "https://app.coolify.io/api/v1/applications/jwgokkggkosokss4ok4wss80/restart" \
  -H "Authorization: Bearer $COOLIFY_TOKEN"

Git-based deploy

Push to the staging branch of jniox/ods-notification-hub.

Note: Coolify GitHub OAuth is broken for private repos. The app uses a public git URL. Manual deploy token or SSH key must be configured in Coolify source settings.

6. How to Check Logs

# On srv-staging (35.195.54.220):
docker logs --tail 200 -f $(docker ps -qf "label=coolify.applicationId=jwgokkggkosokss4ok4wss80")

7. Common Issues and Fixes

Issue: Emails stuck in Pending status

Symptom: Notifications created but never delivered.

Cause: RESEND_API_KEY not set or invalid.

Fix: Set a valid Resend API key in Coolify environment variables. Verify with:

curl -sf -X GET "https://api.resend.com/emails" \
  -H "Authorization: Bearer $RESEND_API_KEY"

Issue: SEC-001 JWT verification

Context: Security review verified RS256 JWT support is working. If JWT verification fails, check that OID JWKS endpoint is reachable:

docker exec <container> curl -sf http://oid:8080/.well-known/jwks.json

Issue: Coolify deploy fails (private repo)

Symptom: Build fails with “repository not found” or authentication error.

Fix: Coolify GitHub OAuth is known broken for private repos. Configure SSH deploy key or personal access token in Coolify source settings.

8. How to Restart

docker restart $(docker ps -qf "label=coolify.applicationId=jwgokkggkosokss4ok4wss80")

9. Dependencies

Dependency Type Details
PostgreSQL 17 Database ods-postgres:5432, schema notifications
Redpanda Event bus redpanda:9092, topic notifications.events
OID Auth JWKS endpoint for JWT verification
Resend External API Email delivery (optional – degrades gracefully)

10. Monitoring / Alerting