03:59 UTC | 19:59 PT
We’re happy to report that the intermittent 502 Gateway Error issues affecting Guide on Pod 14 have been resolved.
03:28 UTC | 19:28 PT
We continue to investigate the intermittent 502 Bad Gateway Errors experienced on Guide on Pod 14.
02:57 UTC | 18:57 PT
We are investigating reports of customers having access issues with Guide on Pod 14.
During this incident, customers on Pod 14 were intermittently receiving 502 errors when attempting to access Help Center. This incident was the result of decreased capacity resulting from resulting from a rapid increase in Help Center requests from certain accounts. We resolved this by implementing additional rate limiting for our DDoS protection system. To prevent further occurrences of this issue, our teams are investigating specific rate limits based on host header domains. In addition, we are further evaluating additional runbook practices that involve our own capacity checks as well as ones based on our DDoS protection system to ensure that we are able to quickly respond to and assess traffic increases of this nature.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.