Incident with multiple GitHub services
Incident
Report for GitHub
Resolved
On June 17, 2025, between 19:32 UTC and 20:03 UTC, an internal routing policy deployment to a subset of network devices caused reachability issues for certain network address blocks within our datacenters.
Authenticated users of the github.com UI experienced 3-4% error rates for the duration. Authenticated callers of the API experienced 40% error rates. Unauthenticated requests to the UI and API experienced nearly 100% error rates for the duration. Actions service experienced 2.5% of runs being delayed for an average of 8 minutes and 3% of runs failing. Large File Storage (LFS) requests experienced 0.978% errors.
At 19:54 UTC, the deployment was rolled back, and network availability for the affected systems was restored. At 20:03 UTC, we fully restored normal operations.
To prevent similar issues, we are expanding our validation process for routing policy changes.
Authenticated users of the github.com UI experienced 3-4% error rates for the duration. Authenticated callers of the API experienced 40% error rates. Unauthenticated requests to the UI and API experienced nearly 100% error rates for the duration. Actions service experienced 2.5% of runs being delayed for an average of 8 minutes and 3% of runs failing. Large File Storage (LFS) requests experienced 0.978% errors.
At 19:54 UTC, the deployment was rolled back, and network availability for the affected systems was restored. At 20:03 UTC, we fully restored normal operations.
To prevent similar issues, we are expanding our validation process for routing policy changes.
Update
Actions is operating normally.
Update
Codespaces is experiencing degraded performance. We are continuing to investigate.
Update
Webhooks is operating normally.
Update
Pull Requests is operating normally.
Update
API Requests is operating normally.
Update
Issues is operating normally.
Update
API Requests is experiencing degraded performance. We are continuing to investigate.
Update
We experienced problems with multiple services, causing disruptions for some users. We have identified the cause and are rolling out changes to restore normal service. Many services are recovering, but full recovery is ongoing.
Update
Copilot is operating normally.
Update
Pages is operating normally.
Update
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Update
Pull Requests is experiencing degraded availability. We are continuing to investigate.
Update
Copilot is experiencing degraded performance. We are continuing to investigate.
Update
Actions is experiencing degraded performance. We are continuing to investigate.
Update
Webhooks is experiencing degraded performance. We are continuing to investigate.
Update
API Requests is experiencing degraded availability. We are continuing to investigate.
Update
We are investigating reports of issues with many services impacting segments of customers. We will continue to keep users updated on progress towards mitigation.
Update
API Requests is experiencing degraded performance. We are continuing to investigate.
Update
Pages is experiencing degraded performance. We are continuing to investigate.
Update
Copilot is experiencing degraded availability. We are continuing to investigate.
Update
Issues is experiencing degraded performance. We are continuing to investigate.
Investigating
We are investigating reports of degraded performance for Copilot
This incident affected: Webhooks, API Requests, Issues, Pull Requests, Actions, Pages, Codespaces, and Copilot.
Subscribe to our developer newsletter
Get tips, technical guides, and best practices. Twice a month. Right in your inbox.
Subscribe