GitHub header

Disruption with some GitHub services

Incident Report for GitHub

Resolved

On August 28, 2024, from 21:40 to 23:43 UTC, up to 25% of unauthenticated dotcom traffic in SE Asia (representing <1% of global traffic) encountered HTTP 500 errors. We observed elevated error rates at one of our global points of presence, where geo-DNS health checks were failing. We identified unhealthy cloud hardware in the region, indicated by abnormal CPU utilization patterns. As a result, we drained the site at 23:26 UTC, which promptly restored normal traffic operations.
Posted Aug 28, 2024 - 23:43 UTC

Update

Our mitigation has taken effect and impact is resolved.
Posted Aug 28, 2024 - 23:43 UTC

Update

Our mitigation is in place and we are seeing a reduction in overall impact. We are continuing to monitor until we are confident that the issue has been resolved.
Posted Aug 28, 2024 - 23:38 UTC

Update

We have identified a potential mitigation and are currently testing.
Posted Aug 28, 2024 - 23:33 UTC

Update

We are experiencing reduced overall impact, but continue to see a continued small impact to unauthenticated requests. We are continuing to investigate.
Posted Aug 28, 2024 - 23:14 UTC

Update

We are continuing to see customer impact and are still investigating.
Posted Aug 28, 2024 - 22:53 UTC

Update

We are continuing to investigate.
Posted Aug 28, 2024 - 22:37 UTC

Update

We are seeing cases of user impact in some locations are continuing to investigate.
Posted Aug 28, 2024 - 22:19 UTC

Update

We are seeing elevated traffic in some of our regions that we are in process of investigating.
Posted Aug 28, 2024 - 22:06 UTC

Investigating

We are currently investigating this issue.
Posted Aug 28, 2024 - 22:02 UTC