Disruption with some GitHub services
Incident
Report for GitHub
Resolved
On February 25th, 2025, between 14:25 UTC and 16:44 UTC email and web notifications experienced delivery delays. At the peak of the incident the delay resulted in ~10% of all notifications taking over 10 minutes to be delivered, with the remaining ~90% being delivered within 5-10 minutes. This was due to insufficient capacity in worker pools as a result of increased load during peak hours.
We also encountered delivery delays for a small number of webhooks, with delays of up-to 2.5 minutes to be delivered.
We mitigated the incident by scaling out the service to meet the demand.
The increase in capacity gives us extra headroom, and we are working to improve our capacity planning to prevent issues like this occurring in the future.
We also encountered delivery delays for a small number of webhooks, with delays of up-to 2.5 minutes to be delivered.
We mitigated the incident by scaling out the service to meet the demand.
The increase in capacity gives us extra headroom, and we are working to improve our capacity planning to prevent issues like this occurring in the future.
Update
Web and email notifications are caught up, resolving the incident.
Update
We're continuing to investigate delayed web and email notifications.
Update
We're continuing to investigate delayed web and email notifications.
Update
We're investigating delays in web and email notifications impacting all customers.
Investigating
We are currently investigating this issue.
Subscribe to our developer newsletter
Get tips, technical guides, and best practices. Twice a month. Right in your inbox.
Subscribe