Incident on 2011-01-04 16:10:00 UTC

Incident Report for GitHub

Resolved

2011-01-04 16:10:00 - One of the file servers failed over temporarily. Working on it...
2011-01-04 17:40:00 - The HA/fencing system on the downed file server is screwed up. Working hard to restore order to the universe.
2011-01-04 18:00:00 - The file server is back online in manual failover mode until we can figure out why it freaked. Most repos are now accessible. Will update when all repos are back online.
2011-01-04 18:14:00 - The site may show 404s or empty repos for the partitions that are still not online. Don't worry there is no data loss. Looking to make those errors less scary.
2011-01-04 20:13:31 - All repos are back online. We will be investigating root cause and solutions to prevent this from happening again. Thanks for your patience, everyone.

Posted Jan 04, 2011 - 16:10 UTC

This incident affected: Git Operations.