We have received a post mortem from Microsoft's Azure Engineering team which details that the root cause of the partial database outage was a misconfiguration in their system. Per Microsoft: "We identified a misconfiguration in our system that have caused control plane to have intermittent hiccups when communicating to instances on tenant rings. The short term mitigation was performed by reestablishing fresh connections, and long term fix will be done by adjusting the configuration. We have finished the short term mitigation and working on the long term mitigation. We apologize for the inconvenience this may have caused to you."
Posted Jul 31, 2019 - 16:55 PDT
Monitoring
At approximately 11:57PM PDT on Saturday July 27th, Microsoft Azure reported the event causing the partial outage as resolved. At this time, all functions are operational. We are continuing to monitor this issue.
Posted Jul 28, 2019 - 08:43 PDT
Update
We are continuing to work on a fix for this issue.
Posted Jul 28, 2019 - 08:39 PDT
Update
We are continuing to monitor and communicate with Azure Support as the partial database outage affecting some GoFormz accounts is investigated and rectified by the Microsoft engineering team. We will continue to update this status page as more information is shared.
Posted Jul 27, 2019 - 20:11 PDT
Update
We’ve identified that the cause of the issue is coming from Microsoft Azure. According to the most recent update from Azure Support, Microsoft engineers are working to resolve a critical situation affecting database performance which may be impacting some GoFormz accounts.
Posted Jul 27, 2019 - 16:06 PDT
Identified
We have identified the intermittent disruptions experienced by a limited set of GoFormz users as an Microsoft Azure issue. We are continuing to work with Azure Support to find the root cause and solution.
Posted Jul 27, 2019 - 13:46 PDT
Investigating
We are currently investigating intermittent disruptions affecting a limited set of GoFormz customers.
Posted Jul 27, 2019 - 10:33 PDT
This incident affected: API (v2), Web App, Mobile Sync API, and Automated Workflows.