Why Scraping Pipelines Break i... Note
DZone.com

Why Scraping Pipelines Break in Production

Production code gets scrutiny. Builds are watched, libraries are checked for known vulnerabilities, and someone is responsible when a service fails at night. The job that pulls in outside data usually escapes all of this. It began as a quick script, got scheduled, and has run untouched ever since. It has no tests, its only error handling is a catch-all exception block, and its author may have left the team long ago.