18:34 UTC | 10:34 PT
We're happy to report that the issue that some customers experienced with errant values in Chat real-time monitoring has been resolved. Please reach out to us if you're still experiencing the issue.
18:26 UTC | 10:26 PT
We're seeing improved results regarding Chat Dashboards at this time. Please reach out and let us know if you continue to see any further issues.
17:38 UTC | 09:38 PT
Our teams are still investigating errant values in some customers' Chat Dashboard. We will post more information when we have it.
17:12 UTC | 09:12 PT
We are currently investigating an issue in which some customers are seeing errant values in their Chat Dashboard
On January 24, 2019 at 14:10 UTC / 06:10 PST, we started receiving reports of Chat Monitor displaying negative values.
This incident was caused by an issue with a third-party real-time data pipeline not consuming all Chat events, causing partial aggregation of events leading to incorrect numbers being reported in the Chat Monitor. During the investigation, our team identified some data pipeline consumers were disconnected and restarted those services resulting in full recovery.
To prevent this issue from occurring in the future, we have scheduled work to improve logging as well as additional monitoring and alerting to respond quicker to data pipeline issues.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.