Why your monitoring is missing the dumbest outages
An expired TLS certificate on an internal subdomain caused a silent outage without any alerts. This occurred because monitoring focused on application errors, neglecting infrastructure components with deadlines. Three domain-level failures can cause outages with no application logs: TLS certificate expiry, domain registration expiry, and DNS misconfiguration. TLS certificates expire and become invalid, causing browsers to show security warnings and APIs to reject connections. Domain registration expiry means the actual domain name lapses, causing DNS resolution to fail entirely. DNS misconfigurations can lead to records not propagating correctly or serving old IP addresses. Bash commands like dig and openssl s_client can check DNS resolution and TLS certificate dates. The openssl command can display a certificate's expiration date, and further commands can calculate the remaining validity period. whois queries provide information about domain registration, including expiration dates and status codes. To prevent such outages, these checks should be automated with cron jobs or monitoring tools like Prometheus. Domain registration should ideally have auto-renew enabled as a primary safeguard. Manually running these checks is useful for ad-hoc verification, but automation is crucial for ongoing reliability. Such outages are particularly embarrassing as they stem from simple oversight rather than complex technical failures.
digandopenssl s_clientcan check DNS resolution and TLS certificate dates. Theopensslcommand can display a certificate's expiration date, and further commands can calculate the remaining validity period.whoisqueries provide information about domain registration, including expiration dates and status codes. To prevent such outages, these checks should be automated with cron jobs or monitoring tools like Prometheus. Domain registration should ideally have auto-renew enabled as a primary safeguard. Manually running these checks is useful for ad-hoc verification, but automation is crucial for ongoing reliability. Such outages are particularly embarrassing as they stem from simple oversight rather than complex technical failures.