Resolved -
This incident has been resolved. Thank you for your patience and understanding as we addressed this issue. A detailed root cause analysis will be shared as soon as it is available.
May 12, 17:43 UTC
Update -
All services have fully recovered.
May 12, 17:43 UTC
Update -
CodeQL has fully recovered. We're continuing to work on recovery for the remaining impacted services.
May 12, 16:59 UTC
Update -
Webhooks have fully recovered. Continuing to work on recovery for the other services.
May 12, 16:29 UTC
Update -
Webhooks is operating normally.
May 12, 16:28 UTC
Update -
We've established that most delays are related to a queuing service and are working to scale out. Early signals from the scale-out are showing signs of recovery for some services. We'll provide an update when services are fully recovered.
May 12, 16:18 UTC
Update -
Webhooks is experiencing degraded performance. We are continuing to investigate.
May 12, 15:44 UTC
Update -
We're continuing to investigate issues with CodeQL actions workflows. We're additionally seeing delays for notifications, webhooks, and the Slack integration.
May 12, 15:42 UTC
Update -
CodeQL actions are currently experiencing delays, which may result in those actions being stuck in a pending state or having failed due to a timeout.
May 12, 15:13 UTC
Investigating -
We are investigating reports of degraded performance for CodeQL
May 12, 14:38 UTC