10:10 GMT+1 / 02:10 PST:
We are investigating reports of data syncing issues with Insights. More info to follow
10:32 GMT+1 / 02:32 PST:
We are continuing to investigate reported issues with Insights. Next update within 30 minutes
11:07 GMT+1 / 03:07 PST:
Our Operations team have narrowed down the source of problems affecting Insights and are working to restore the service
11:34 GMT+1 / 03:34 PST:
We have resolved the issues impacting Insights. Post-mortem to follow shortly
This incident was caused by a code issue in the latest deploy for our NPS feature. The bad code caused GoodData API requests to NPS to fail authentication, resulting in 401s as response and some data in Insights not loading.
To mitigate the incident, we reverted the code, after which Insights began to load again.
Aside from working to improve our QA for deploy code, we're looking into introducing additional alerts for NPS incremental API, better integration testing and working with GoodData to ensure alignment on our supported authentication protocols.
FOR MORE INFORMATION
Please subscribe to this article for regular updates until the issue is resolved. If you aren't subscribed to our Twitter feed, we encourage you to do so in order to get the most current information about any service issues. We also record all site outages on our system status page where you can see the past 12 months of service uptime. If you have questions about this issue, please open a ticket with us by sending a note to firstname.lastname@example.org.