Project case study

Event-Driven Workflow Scheduler

Dual-path event scheduler for real-time and future-dated triggers. Push path for immediate domain events, pull path via a managed scheduler for precise timestamps, with a dedicated failure handler for safe recovery.

Private implementation; details are shared at an architectural level.

Problem

Existing scheduler only supported fixed intervals and a single daily polling pass
No way to trigger work on a real-time domain event
No precision on future-dated triggers
Critical time windows needed manual babysitting
Data pre-calculated before execution went stale by the time it ran

Constraints

Scheduling drives real user-facing operations, a missed run is a real miss
Has to support real-time and future-dated triggers in one coherent design
State changes between scheduling and firing, so pre-resolving entity lists is not safe
Failures must be visible and recoverable, not silent
Has to coexist with the existing downstream workers, not replace them

What We Tried First

First instinct was to pre-resolve the full entity list inside the daily pre-calculation pass
Simpler pipeline, easier to test, fewer moving parts
Did not survive first contact with real state changes between pre-calc and execution
The scheduler was firing on a stale snapshot of the world
That was the point where entity resolution had to move to execution time and the dual-path design stopped being optional

Architecture

Two deliberate paths, separated on purpose: push for real-time triggers, pull for future-dated
Push path wires domain events straight to the existing message broker
Pull path uses a managed scheduler with one schedule per task, not one per entity
Entities are resolved at execution time, then fanned out to the existing downstream workers unchanged
A dedicated failure handler watches the dead-letter queue, captures context, and allows controlled re-queuing

Tradeoffs

Execution-time entity resolution over pre-calculation. Simpler to pre-calc but stale data is worse than no data
One schedule per task over one per entity. Keeps quota pressure bounded and the next-run update atomic
Reuse the existing message broker instead of rewriting downstream workers. Coexistence beats big-bang migration
Minimal payloads at the scheduling boundary, entity fan-out after. Keeps scheduling cheap and entity logic centralised
Dedicated failure handler over best-effort logging. Silent failures in a scheduling system are the worst kind

Outcome

Scheduling moved from once-daily polling to exact-time delivery
Failures are now visible and recoverable instead of silent
Downstream workers still run unchanged
Same pattern now powers multiple scheduled workloads