Azure-based Network Outages
Incident Report for GoFormz
Resolved
This incident has been resolved. Please see below message from the Microsoft Azure support team:

Multiple Services - Mitigated
Summary of Impact: Between 00:57 and 03:40 UTC on 20 Nov 2019, customers and services utilizing Azure Front Door (AFD) services were impacted by an infrastructure service failure. This resulted in a loss of connectivity to multiple services reliant on AFD.

Azure Front Door services provide Edge caching and network entry point services to the Microsoft global network. This issue impacted a large percentage of Microsoft Services, though not all services were impacted. Many impacted services were able to initiate fail over from the AFD platform, providing immediate mitigation to their customers.

Preliminary Root Cause: During a recent periodic deployment, initial safety checks did not detect the issue and prevent the roll out. Monitoring detected once service failure was experienced and alerted engineers.

Mitigation: Engineers immediately initiated deployment rollback procedures to correct the underlying Azure Front Door issue. This was completed at 02:40 UTC on 20 Nov 2019, at which point Impacted services began recovering.

A detailed root cause analysis will be published within 72 hours.
Posted Nov 20, 2019 - 08:53 PST
Monitoring
At 5pm Pacific Time GoFormz was impacted by Azure CDN outages causing problems with login for approximately 45 minutes. The CDN outages have been traced to a current issue with Microsoft Azure, our cloud services provider. Azure engineers have identified outages of their Network Infrastructure across all regions.

Even though GoFormz services remain stable, we will continue to monitor all services and will update if our status changes.

Current Message from Azure Status as of 7:50pm Pacific Time:

Multiple Services - Applying Mitigation
Summary of Impact: Starting at 00:57 UTC on 20 Nov 2019, a subset of customers may experience failures when attempting to access Azure Resources globally.
Engineers have identified a root cause and mitigation is in progress; customers may see signs of recovery as this progresses.
Posted Nov 19, 2019 - 20:01 PST
This incident affected: Web App and Mobile Sync API.