Armin Ronacher: Absurd Workflows: Durable Execution With Just Postgres

The author is building agents and addressing the challenge of durable execution, a crucial aspect of reliable long-running functions. They created "Absurd 1," a SQL-only library built on Postgres for durable workflows to avoid third-party dependencies. Durable execution involves state management and task division into smaller steps, ensuring resilience to failures. Postgres's queuing and state storage capabilities are leveraged using `SELECT FOR UPDATE SKIP LOCKED` for task queues and checkpointing. Absurd is a single SQL file using SDKs for ease of use, dividing tasks into steps executed sequentially. Tasks can be suspended or fail, restarting at the failed step with stored checkpoints. Events and sleeps are also supported for more complex workflows. Absurd's agents have a single step iterating until completion, used in an example agent with message handling. This approach provides checkpoint and retry mechanisms for task recovery and agent progress. The system allows for event waiting and emission to facilitate coordination with other processes. Absurd's simplicity, due to not requiring separate services, makes it straightforward to use and self-host. The author emphasizes that durable workflows, while powerful, can be implemented with minimal complexity using Postgres.