Partial Disruption of GoFormz Services

Incident Report for GoFormz

Postmortem

Essential components of the GoFormz platform became unresponsive on Friday 12:25pm Pacific Time due to a corrupt configuration change in our Authentication service. After identifying the underlying issue, GoFormz engineering rolled back the change and restarted impacted services. Upon service recovery GoFormz’s primary authentication database required upscaling to recover completely. All GoFormz services have returned to normal levels as of 1:20pm Pacific Time.

GoFormz engineering will be reviewing and adjusting these configuration change procedures to mitigate future occurrences.

Posted Apr 17, 2020 - 15:59 PDT

Resolved

This incident has been resolved.

Posted Apr 17, 2020 - 15:57 PDT

Monitoring

We have identified the issue and implemented a fix. As of approximately 1:20 PM PDT, GoFormz services have returned to normal functionality. We are continuing the closely monitor this issue.

Posted Apr 17, 2020 - 13:36 PDT

Identified

The issue has been identified and a fix is being implemented.

Posted Apr 17, 2020 - 13:26 PDT

Investigating

GoFormz is currently experiencing degraded performance. This primarily affects accounts using the GoFormz Workflow or API, and users not already logged into a GoFormz account through the mobile or web app.

Posted Apr 17, 2020 - 12:44 PDT

This incident affected: API (v2), Web App, Mobile Sync API, and Automated Workflows.