10:46 UTC | 03:46 PT
We are happy to report that the performance issues impacting Pod 14 customers are now resolved.
10:08 UTC | 03:08 PT
We’re seeing improvements on POD14 accounts and our Operations team are working on mitigating the impact.
09:31 UTC | 02:31 PT
Our operations team are working to mitigate performance impact on our support product for Pod 14 customers.
09:15 UTC | 02:15 PT
We are experiencing a service disruption on Pod 14 with the Support product. More info to follow.
During this incident, customers experienced very slow performance issues using the Support product, which we pinpointed as originating with Embeddables servers. We immediately began to restart the Embeddables server on Pod 14 in order to stabilize the situation. Before the restarted server was ready to be rotated back into production, the situation improved and customers reported that their issues disappeared. After some investigation, it was clear that the Embeddables issues were further symptoms of the situation, than the root cause. Evidence now points to a load issue as the root cause of this service incident, so additional app servers have been put into production for both pod13 and 14. Investigations into this issue are continuing.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.