Incident with Webhooks
Incident
Report for GitHub
Resolved
On August 12, 2024 from 13:39 to 14:28 UTC some users experienced an elevated rate of errors of up to 0.45% from the GitHub API. Less than 5% of webhooks interactions failed and less than 0.5% of Actions runs were delayed.
This impact was caused by internal networking instances being insufficiently scaled.
We mitigated the incident by provisioning additional instances. We are working to enhance the sizing strategy for the relevant infrastructure to prevent similar issues and to also improve monitoring and processes to reduce our time to detection and mitigation of issues like this one in the future.
This impact was caused by internal networking instances being insufficiently scaled.
We mitigated the incident by provisioning additional instances. We are working to enhance the sizing strategy for the relevant infrastructure to prevent similar issues and to also improve monitoring and processes to reduce our time to detection and mitigation of issues like this one in the future.
Update
Webhooks is experiencing degraded performance. We are continuing to investigate.
Update
Webhooks is operating normally.
Update
Some customers may see failures calling the webhooks API or running queries for enterprise or organization audit logs. We have started mitigating the issue and are watching recovery. We will provide an update within the next 30 minutes.
Investigating
We are investigating reports of degraded performance for Webhooks
This incident affected: Webhooks.
