Automated cleanup jobs for retrying outbound webhooks can become unexpectedly resource-intensive as data grows. A simple nightly delete job might fail to keep up with increased volume. To manage this, a cron job should trigger a public HTTP endpoint for initiating cleanup tasks. For substantial deletions, this endpoint should delegate the work to idempotent queue workers that process data in bounded batches. The retry ledger is crucial, ensuring that recurring cleanup actions do not cause unintended destructive operations by using unique batch keys for conditional deletion.Workers must be designed to handle duplicate messages safely, treating them as no-ops. This idempotency is vital, as retries are normal and the system must robustly handle them. A Node.js cron HTTP endpoint can protect against duplicate work by quickly publishing a single, bounded batch with an idempotency key. The actual deletion is then handled by a dedicated worker, not the cron job itself, which has time limitations.The queue worker should be intentionally simple, processing one bounded batch, deleting records, and acknowledging completion only after a successful transaction. This design makes retries harmless. Implementing a dead-letter queue is recommended for persistent failures, providing a way to inspect and redrive problematic messages. Testing restart scenarios is important to ensure the idempotency mechanism functions correctly under various failure conditions.Observability is key, with detailed logging of batch IDs, counts, and statuses. Avoid placing large amounts of data in queue messages. For complex, multi-step cleanup processes, workflow engines like Temporal or Airflow are more suitable. The described pattern is best for simpler, single-pass retention tasks and requires publicly accessible endpoints.The operational loop involves triggering, enqueuing, claiming, deleting, recording, and acknowledging. Regularly monitoring key metrics like unprocessed batch age and dead-letter queue depth is essential. Running a dry-run query before changing retention policies provides a safety net against accidental data loss. This entire process should be simple enough to understand and manage even during off-hours.