I got out of GitHub last year and self hosted a vps that runs gitea.
My work llm provider is GitHub copilot and we are completely blocked since morning. My question is who’s comparable to copilot vscode and cli. Multi model is table stakes.
The most shocking thing to me about the entire GitHub fiasco is: No matter how bad the uptime is, no matter how bad the UI is. We will keep using the software / service we depend on.
Why do we as software engineers have set fault tolerance and high availability as the primary goal of any software / infrastructure architecture? This shows it's completely irrelevant.
Personally, Fastcomments and all my projects can easily be switched to a self hosted provider, keeping GitHub as a read only mirror. It's probably a couple days work with today's tools.
It is not. I am just not happy with the alternatives; Gitlab, and Codeberg, for well-discussed reasons. I will pounce as soon as the right one comes along, and I have a few on my radar.
I'm raising $1MM for 10% of ShipGit. We're a headless code forge committed to 99.9% uptime that integrates with agents via MCP, Linear, and popular CI/CD tools.
> Update - We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available
It seems vastly higher on every region and VPN I've tested, so I think this is a case of the "technicalies".
It is technically 20%, because a bunch of the requests that happen on page load do succeed. Just not the few crucial ones that are required for the page to load correctly - those return a 500 the vast supermajority of the time.
Feel free to disregard this because ultimately its a tangent that doesn't matter; but supermajority is specifically a political term referring to the required threshold to advance legislation when simple majority isn't procedurally enough. It doesn't mean simply a greater share of.a majority
For supermajority: the construction means “beyond majority” or “above majority”, it is implemented as a higher percentage in most systems, and there is a utility in being able to refer to high ratio majority.
So people have repurposed a technical word to a general word, of roughly the same meaning.
It's neither. If you accept and approve of people wearing their pants around their knees, its a fashion evolution. If you think pants should cover your ass, it's just poor behavior. Analogous to the use of language.
The error rate is 100% on the service that allows you to create PRs in the web site. Actual remote git service is working just fine. I might try the API to make a PR, see if that's working.
Update: yes that worked. gh CLI using the API was able to make a PR while the web interface was inoperable.
Second update: defeat from the jaws of victory -- the page that displays open PRs is also down.
Third update: CI isn't working anyway, so the CLI PR creation workaround doesn't achieve much.
GitHub can’t focus on the basics like a stable platform. However, when they release stack commits a few weeks ago somebody had time to make the hamburger menu an unprofessional stack of pancakes…cute.
While i get the sentiment and the frustration, I highly doubt an SRE worked on the hamburger menu.
In my experience some teams move a lot faster than others and that leads to asymmetric quality. If all devs could pick up a new language and skillset overnigh then companies would have a lot more flexibility in their roadmaps, but humans don't scale that way - with or without AI.
This weekend I managed to get a test into some sort of infinite loop that didn’t overflow or time out, so I had to single-step in to figure out where the last line it broke on. Something to do with events propagating. Oh and someone misunderstanding an unfamiliar mocking framework and not preserving someone else’s cleanup code. I won’t name and shame but his name stars with an h and ends with inkley. Mangled someone else’s abandoned PR thinking one cleanup function was a superset of another. It was not, dumdum.
Event systems have interesting ways to fail that resist throwing good engineering at them to fix. For a long time I refused to have anything to do with them, but now we have whole companies built on them and also things like React and what can you do? There are harmonic failure modes where the system doesn’t fail but an event keeps echoing forever and using up 50% of the available compute before anyone notices it’s been like that for months and slowly getting worse. And then there’s the Poison Pill where a message crashes a process, then gets picked up by the next and kills that one too until they’re all dead.
Funny because I spent a couple of years on self-hosted Gitea, including doing some work on Gitea itself to add features we wanted. Then I changed projects and thought "well that was a lot of work, let's just do what everyone does and use GitHub". Now twiddling my thumbs this morning...wondering if I need to resurrect the Gitea deployment.
This is pretty much why I started Fjord. I like Forgejo/Gitea, but didn't think “run your own forge forever” should be the only alternative to GitHub. We run dedicated Forgejo instances for people who want that middle ground.
Well I was dumped out of Github Copilot as this issue started, and haven't been able to get back in. Seems like a slightly strange thing to be connected, but feasible.
I find it quite amusing what they seem to consider "degraded performance" to mean -- I guess if you really squint, your service not working at all is actually just very large latency spikes (until the service recovers) and thus can be labelled as "services experiencing degraded performance for a small portion of our customers".
I got out of GitHub last year and self hosted a vps that runs gitea.
My work llm provider is GitHub copilot and we are completely blocked since morning. My question is who’s comparable to copilot vscode and cli. Multi model is table stakes.
The most shocking thing to me about the entire GitHub fiasco is: No matter how bad the uptime is, no matter how bad the UI is. We will keep using the software / service we depend on.
Why do we as software engineers have set fault tolerance and high availability as the primary goal of any software / infrastructure architecture? This shows it's completely irrelevant.
Personally, Fastcomments and all my projects can easily be switched to a self hosted provider, keeping GitHub as a read only mirror. It's probably a couple days work with today's tools.
It is not. I am just not happy with the alternatives; Gitlab, and Codeberg, for well-discussed reasons. I will pounce as soon as the right one comes along, and I have a few on my radar.
I can’t help but think of how AWS didn’t go through with killing CodeCommit and how happy that user base is to not be on GitHub.
Cursor Origin launched today: https://news.ycombinator.com/item?id=49334209
I'm raising $1MM for 10% of ShipGit. We're a headless code forge committed to 99.9% uptime that integrates with agents via MCP, Linear, and popular CI/CD tools.
Is github down? Does the day end in y?
No, es lunes.
Damn. Wonder why it's not working then
> Update - We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available
brutal
20%, do we just make numbers up?
It seems vastly higher on every region and VPN I've tested, so I think this is a case of the "technicalies".
It is technically 20%, because a bunch of the requests that happen on page load do succeed. Just not the few crucial ones that are required for the page to load correctly - those return a 500 the vast supermajority of the time.
I can't get a single PR or issue to work. Maybe it's "20% globally, 100% anywhere people are awake".
Feel free to disregard this because ultimately its a tangent that doesn't matter; but supermajority is specifically a political term referring to the required threshold to advance legislation when simple majority isn't procedurally enough. It doesn't mean simply a greater share of.a majority
This is how language evolves.
For supermajority: the construction means “beyond majority” or “above majority”, it is implemented as a higher percentage in most systems, and there is a utility in being able to refer to high ratio majority.
So people have repurposed a technical word to a general word, of roughly the same meaning.
No, some usage is just wrong, even if many people are doing it.
English is descriptive not prescriptive. The meaning of words change over time.
"Disappoint" was originally the inverse of "appoint" and meant to remove from office.
It's neither. If you accept and approve of people wearing their pants around their knees, its a fashion evolution. If you think pants should cover your ass, it's just poor behavior. Analogous to the use of language.
20% of requests on a GUI page also makes the whole thing unusable.
Depends if this is coming from engineering or management
The error rate is 100% on the service that allows you to create PRs in the web site. Actual remote git service is working just fine. I might try the API to make a PR, see if that's working.
Update: yes that worked. gh CLI using the API was able to make a PR while the web interface was inoperable.
Second update: defeat from the jaws of victory -- the page that displays open PRs is also down.
Third update: CI isn't working anyway, so the CLI PR creation workaround doesn't achieve much.
GitHub can’t focus on the basics like a stable platform. However, when they release stack commits a few weeks ago somebody had time to make the hamburger menu an unprofessional stack of pancakes…cute.
While i get the sentiment and the frustration, I highly doubt an SRE worked on the hamburger menu.
In my experience some teams move a lot faster than others and that leads to asymmetric quality. If all devs could pick up a new language and skillset overnigh then companies would have a lot more flexibility in their roadmaps, but humans don't scale that way - with or without AI.
This weekend I managed to get a test into some sort of infinite loop that didn’t overflow or time out, so I had to single-step in to figure out where the last line it broke on. Something to do with events propagating. Oh and someone misunderstanding an unfamiliar mocking framework and not preserving someone else’s cleanup code. I won’t name and shame but his name stars with an h and ends with inkley. Mangled someone else’s abandoned PR thinking one cleanup function was a superset of another. It was not, dumdum.
Event systems have interesting ways to fail that resist throwing good engineering at them to fix. For a long time I refused to have anything to do with them, but now we have whole companies built on them and also things like React and what can you do? There are harmonic failure modes where the system doesn’t fail but an event keeps echoing forever and using up 50% of the available compute before anyone notices it’s been like that for months and slowly getting worse. And then there’s the Poison Pill where a message crashes a process, then gets picked up by the next and kills that one too until they’re all dead.
We're moving to self hosted Gitea. It's never been easier to do devops with the help of LLMs. I cannot believe how flaky GitHub is.
https://git.tinygrad.win/tinygrad/tinygrad
Funny because I spent a couple of years on self-hosted Gitea, including doing some work on Gitea itself to add features we wanted. Then I changed projects and thought "well that was a lot of work, let's just do what everyone does and use GitHub". Now twiddling my thumbs this morning...wondering if I need to resurrect the Gitea deployment.
This is pretty much why I started Fjord. I like Forgejo/Gitea, but didn't think “run your own forge forever” should be the only alternative to GitHub. We run dedicated Forgejo instances for people who want that middle ground.
I have a habit of reading the source code via github web interface instead of cloning and reading it in a much more comfy text editor.
Today I was forced to clone one because of "the incident". Which is a good thing.
incident was just created - https://www.githubstatus.com/incidents/zkxwbgr0cnmx
heaven forbid I want to commit some code this morning
Well I was dumped out of Github Copilot as this issue started, and haven't been able to get back in. Seems like a slightly strange thing to be connected, but feasible.
Does anyone know what could be the cause of this?
LLM code or just terrible management?
Yes it is https://downdetector.com/status/github/
Cannot retrieve latest commit at this time. :<
Seems to affect web as well. Currently working on a PR and now it says "Merge status cannot be loaded"
https://www.githubstatus.com/
It's all green which means it's working. Ignore the unicorn error on every page.
edit: After 5-10 minutes they got around to opening an incident and (apparently manually?) marking the API as having 'degraded performance'.
I find it quite amusing what they seem to consider "degraded performance" to mean -- I guess if you really squint, your service not working at all is actually just very large latency spikes (until the service recovers) and thus can be labelled as "services experiencing degraded performance for a small portion of our customers".
Even better "degraded performance" doesn't seem to count against their uptime stats. Issues is labeled as "degraded" with 100% uptime.
Not sure if related, but claude seems to be down for me too.
hmm, I would have thought they could make github 1000x more reliable with llm's
Same here. Just as I was going to push out a release
How are those layoffs treating you, Microsoft?
"degraded performance" is the new 2026 term to say that your platform is down unusable.
On my side it is funny,I'm encountering the following errors: "no server available to process your request".
Maybe azure is out of capacity in its datacenters... Or they didn't pay the server bill.
Another day, another github incident
Clones, repo browsing etc are all working.
/pull/:id/changes is not working, but /pull/:id.diff _is_. It seems parts of the API is also ok? some gh commands I tried worked.
Files changed works on the mobile app somehow. I guess a particular api version or something is down.
Anyway if someone needs files changed and merge status, check the mobile app. Past actions are also loading on that.
[Android, India]
Some more discussion: https://news.ycombinator.com/item?id=49330684
Been doing this wrong for years, thanks
“Can’t get latest commit”
Ffs GitHub, get your act together.
reminder: git is distributed