Skipper, Airbnb’s embedded wor... Note

Skipper, Airbnb’s embedded workflow engine, is now open source

Skipper is an open-source, embedded workflow engine that runs directly within a service, eliminating the need for a separate orchestration cluster. It allows developers to define multi-step business processes as Java or Kotlin classes. Skipper persists workflow progress, enabling recovery from crashes and transient failures by reusing existing service dependencies. The engine's design was driven by Airbnb's desire for services to own their workflows without relying on shared orchestration infrastructure.In Skipper's architecture, service replicas host the workflow runtime and scheduler, sharing a database for workflow state. If a replica fails, another can take over its work, with takeover timing determined by scheduling and lease settings. This integrated approach reduces deployed systems and places workflow execution responsibility with the service's team. Workflows are represented as classes, with "actions" performing specific tasks and recording their results for recovery purposes.The engine supports features like waiting for external events via signals, with the ability to define timeouts. Idempotency must be handled by the actions themselves, as Skipper does not perform transactional calls to downstream systems. Recovery occurs through checkpoints and replay, where interrupted workflows re-execute from the start, utilizing saved action results. Deterministic workflow code is crucial for this replay mechanism, with I/O and randomness confined to actions. Developers can choose immediate checkpointing to minimize re-execution of completed actions after a crash.Skipper is well-suited for Java or Kotlin services managing multi-step processes that require durable execution within their existing deployments. The open-source release includes the core engine, database adapters, test utilities, and an admin UI. Future plans involve expanding storage plugin support and improving tooling for workflow evolution and debugging. Contributions are welcomed for automated compatibility checks and replay visualization.
CdXz5zHNQW_M8DQeN2ZaU.png