We deployed a new configuration to improve our network availability. This resulted in a small percentage of user traffic getting incorrectly blocked, but missed by our automated detections. We mitigated this with a change to the configuration, rolled out slowly over the last hour of this incident time for safe deployment. Beyond the learnings related to the config, we are analyzing how we can more quickly detect this kind of impact as part of future configuration rollouts.
Posted Oct 06, 2023 - 02:59 UTC
Update
We have confirmed that the fix has resolved the issue in the subset of regions where it has been deployed. We are now continuing the deployment to the remaining regions.
Posted Oct 06, 2023 - 02:42 UTC
Update
We are monitoring the rollout of the fix and are beginning to see signs of improvement. We will send another update shortly.
Posted Oct 06, 2023 - 02:03 UTC
Update
A small number of customers are experiencing 403 errors when attempting to access repository data via the API. We have found what we believe to be the cause of the issue and are deploying a fix.
Posted Oct 06, 2023 - 01:31 UTC
Investigating
We are investigating reports of degraded performance for API Requests
