2020-December-14 Service Incident
Postmortem

Dates:

Monday December 14, 15:20 - 23:30 CET

What happened:

Automated Unified Platform tests were not listing Commands on the Sauce UI for ~8 hours.

Why it happened:

A deployment for one of our services to a new pipeline setup was incomplete and this made the service non-functional, which prevented the UI from having access to the lists of Commands.

How we fixed it:

The deployment of the service was reverted to the old pipeline setup.

What we are doing to prevent it from happening again:

We are making sure we have the correct monitoring in place for these kinds of changes to our deployment pipelines.

Posted Dec 21, 2020 - 10:32 CET

Resolved
Automated Unified Platform tests were not listing Commands on Sauce UI today between 6:30 a.m. - 2:30 p.m. PST. We have identified and fixed the problem. All services are fully operational.
Posted Dec 14, 2020 - 13:30 CET