As of 01:30 UTC / 17:30 PST we're investigating the following service incident:
We're currently investigating reports of an issue impacting our views. More information to follow.
02:05 UTC / 18:05 PST:
We are happy to report that the issue involving Ticket Views has been fixed. A Post Mortem will be available shortly.
This incident began when a refactor of some old code around rule condition optimizations was left hanging, with a feature set that had been fully rolled out in a staging environment but never turned on in production. This led us to believe that it was safe to remove a code path that was still providing functionality in a certain case (views that query against multiple drop downs). It was, in fact, not safe and led to some tickets not appearing in views for multiple customers. The problem was ultimately fixed by redeploying an earlier version.
The problem seemed to linger undetected for some time post-deploy, as the initial production datacenter for all of our deploys only has a handful of active customer accounts on it and, therefore, did not have an active account with the proper configuration that would have experienced this problem, so we didn't get an advance warning of a breakage.
As a result of this incident, we have already added tests to QA that exercise the code paths to verify if there's a functionality reliant on them. We're also re-evaluating how we do deploys, including looking at automating some parts of rollouts.
FOR MORE INFORMATION
Please subscribe to this article for regular updates until the issue is resolved. If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues. We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to firstname.lastname@example.org.