2019-04-04 Service Incident
Incident Report for Sauce Labs European Data Center


April 3rd, 2019 11:46 PM - April 4th, 2019 02:39 AM PDT

What happened:

Customers of the Real Device Cloud in the EU were unable to execute any type of automated tests or perform Live Testing.

Why it happened:

Firewall rules pushed to our DNS servers in the EU for security reasons that worked in our other regions caused issues in the EU due to environmental differences.

How we fixed it:

The broken firewall rules were hot patched and the Real Device Cloud in the EU was redeployed.

What we are doing to prevent it from happening again:

The DNS servers in the EU will be rebuilt to match the configuration of DNS servers in other regions our service supports. This will cause no downtime for customers.

Posted 4 months ago. Apr 12, 2019 - 01:46 CEST

The problem has been resolved. All services are fully operational. Please contact support@saucelabs.com if you continue to experience issues.
Posted 5 months ago. Apr 04, 2019 - 12:05 CEST
Android Real Devices continue to operate normally. Our engineers have identified the problematic components causing iOS Real Device issues, and are working on remediation.

All US Devices are functioning normally.
Posted 5 months ago. Apr 04, 2019 - 11:04 CEST
The problem has been corrected and service for Android devices is restored. We continue to investigate a problem with iOS devices.
Posted 5 months ago. Apr 04, 2019 - 10:04 CEST
All EU Real Devices are Offline. Customers may experience problems with the REST API. Our developers are investigating.
Posted 5 months ago. Apr 04, 2019 - 09:17 CEST
This incident affected: Automated RDC Testing (Automated RDC Testing) and REST API (REST API RDC).