18:19 UTC | 11:19 PT
The issues affecting the Connect API have been resolved. If you're still experiencing issues, please let us know.
17:40 UTC | 10:40 PT
The issues affecting the Connect API are stable. We are continuing to monitor the situation as it improves.
17:06 UTC | 10:06 PT
We have identified the root cause of the issues affecting the Connect API. Thank you for your patience as we continue monitoring the situation and verify improvement in performance.
16:42 UTC | 09:42 PT
Our Ops team is still investigating issues related to the Connect API. Thank you for your patience as we continue working through this.
16:18 UTC | 09:18 PT
We’re currently investigating reports of customers experiencing errors when using the Connect API and being unable to log in to the Connect dashboard. We apologize for the inconvenience.
During this incident, Connect users were unable log into the connect dashboard and API calls would return 5xx errors. This resulted in significant performance issues, making the Connect product unusable due to slow loading screens or being logged out due to the 500 errors. Our engineers restarted the API servers which provided temporary relief but the issue surfaced again. A recent deploy was then rolled back which resolved the issue. The root case was a combination of a latent escaped software defect that was only surfaced with the recent code deploy. Our developers have deployed fixes for the defect and have also limited the number of concurrent connections to prevent this from happening again.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.