19:36 UTC | 12:36 PT
Search index issues have been resolved for all impacted customers.
19:12 UTC | 12:12 PT
We have identified the search index issue impacting some customers, and it is now being resolved. Fully normal operation to resume shortly.
18:40 UTC | 11:40 PT
We're investigating reports of stale searches for tickets created today -- mostly for some customers on our West Coast data center.
During this incident, search services in the West Coast Datacenter failed to return results on tickets, organizations, groups, and other search objects created after 13:05 UTC. Objects created prior to 13:05 UTC remained searchable.
This issue was caused by a code change that was intended to run in our testing environment, but instead ran in our production environment. The testing environment contained the correct schema for the change while the production environment did not. To resolve the issue, the correct schema was added to our production environment. Search services were impacted briefly, but soon returned to normal.
FOR MORE INFORMATION
Please subscribe to this article for regular updates until the issue is resolved. If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues. We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to firstname.lastname@example.org.