What happened?
On June 12th, 2025, at approximately 11:29 AM Pacific, the xMatters internal monitoring tools alerted to an issue in the North America region with the xMatters integration platform. While the issue was in progress, some xMatters customers may have experienced workflows not executing correctly or signals not being accepted as expected.
Why did it happen?
The issue occurred because multiple Google Cloud and Google Workspace products experienced external API issues for about three hours between the hours of 10:49 AM to 1:49 PM Pacific.
How did we respond?
As soon as the xMatters monitoring tools reported an issue with the xMatters integration platform, the xMatters Customer Support team initiated the internal Major Incident Management process and engaged the Engineering and incident response teams. The teams quickly determined that the issues were due to problems with an upstream provider and immediately opened a support case for them to investigate. They continued to work with the provider throughout the incident and confirmed with their engineers as they recovered the underlying dependencies.
What are we doing to prevent it from happening again?
The xMatters Support and Engineering teams requested an RCA and prevention methods from the upstream provider. In their response, they confirmed that they will be modularizing their Service Control architecture to prevent failures from affecting the entire service.