Resolved -
This incident is considered resolved.
Oct 14, 18:48 UTC
Monitoring -
Our latest mitigation efforts are proving to be successful. Errors have been dramatically reduced and we are not operating under normal parameters.
We will continue to monitor the situation closely.
Oct 9, 18:38 UTC
Identified -
Sadly our previous efforts were not enough to fully resolve the issues. We are applying further remediation efforts. The system will need to work through a backlog of processing but we are hopeful that service will be back to normal operating parameters soon.
More information to come.
Oct 8, 20:01 UTC
Monitoring -
Performance enhancements have been applied to the data management clusters responsible for the Forecasting tool. UI and API errors are now back to normal operating parameters.
We will monitor the situation closely.
Oct 7, 21:19 UTC
Identified -
The source of the errors has been identified as being a backlog of queries which had accumulated in our data management systems. While the backlog has been reduced and the errors should have ceased, we are working on mitigation measures to reduce the possibility of such occurrences in the future.
More information to come.
Oct 7, 18:02 UTC
Investigating -
We are currently investigating an issue where users may experience errors when running Forecasting queries in the UI and also getting 500 errors when sending API queries tot he avails API.
More information to come.
Oct 7, 16:35 UTC