19:46 UTC | 11:46 PT
We are happy to report that the issue causing search issues in Guide on Pod 13 have been resolved. We are investigating root cause and will have more details in the official post-mortem. If you see any further errors please reach out.
18:52 UTC | 10:52 PT
We saw a brief period from approximately 11:44-12:30 CST where Guide search was failing on Pod13. It has since recovered and searches will complete but we continue to investigate the underlying cause. If you continue to see errors please reach out.
During this incident, customers reported issues with using search functionality in Support and Guide on pod 13.
Due to an authentication issue in our search service, search requests were blocked leading to failed retrieval of search results. Our engineers identified an issue that arises after long periods of no server restarts. After performing a controlled restart, search services resumed.
To prevent this issue from occurring in the future, our team has upgraded the web server engine, scheduled regular invisible restarts, implemented additional monitoring/alerts and scheduled migration to our new search service.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.