We are investigating reports of degraded performance.
Incident
Report for GitHub
Resolved
On January 28, 2024, between 01:00 UTC and 14:00 UTC the Avatars service was degraded and could not return all avatar images requested by users, instead it would return a default, fallback avatar image. This incident impacted, at peak time 6% of the requests for viewing Avatars. Requests that were impacted did not prevent the users from continuing to use any GitHub services. This was due to an issue with the Avatars service connecting to a database host.
We mitigated the incident by restarting the malfunctioning hosts that were not able to return the user avatar images.
We are working to improve alerting and monitoring of our services to reduce our time to detection and mitigation.
We mitigated the incident by restarting the malfunctioning hosts that were not able to return the user avatar images.
We are working to improve alerting and monitoring of our services to reduce our time to detection and mitigation.
Update
We have mitigated all customer impact. We are no longer serving fallback avatar icons when loading web pages for some customers. We continue to monitor the results.
Update
A fix has been implemented for customers seeing the default avatar (octocat) when loading web pages and we are monitoring the results.
Update
Some requests for getting the Avatars are returning the fallback response instead of the asked avatar since they are having issues connecting to the Mysql host
Investigating
We are currently investigating this issue.
