A single DNS race condition brought AWS to its knees
A single DNS race condition brought AWS to its knees
go.theregister.com
A single DNS race condition brought AWS to its knees
Comments
102-
-
oh sure, when they fuck up DNS it's a "race condition".
when I fuck up DNS it's a "fireable offense".
-
It's funny aws report didn't mention 40% sysops were replaced by AI. https://blog.stackademic.com/aws-just-fired-40-of-its-devops-team-then-let-ai-take-their-jobs-d9db9d298bfa
-
-
That's what you get when you let go hundreds of employees from your cloud computing unit in favour of AI.
I hope they end up having to compensate all the billions of losses they caused to all the businesses and people.
-
Consequences? For Amazon?
lol… lmao even
-
They do have contracts and are obligated to provide a certain "up time", which is usually 99% or so. If they fail to provide that, they are liable to compensate for the losses.
Or do you think that Amazon is above the law and no other company could sue them?
It all depends on what kind of contracts they have.
-
They have ORANGE ass makeup on their lips. How did THAT get there???
-
-
Mistakes happen with or without AI
The problem is that the current internet is structured in a way that creates high risk systems that can cause a massive outage. We went from having thousands of independent companies to a handful of massive ones. A mistake by a single company shouldn't be able to black out half the internet.
-
Was it proven that AI wa the cause?
In not saying it wasn't, just that if it really was, I'd like a source for that claim
-
There was an article in my lemmy all feed yesterday claiming so. But it was a super questionable shady site, which people were calling out.
-
No, but it clearly wasn't the solution. They likely could have used some of those people they fired for that.
-
-
Silly peon rich people don't suffer consequences.
-
That's what you get when you let go hundreds of employees
OK but then... what happens when their boss jerk fires hundreds of thousands?
-
-
I DNS see that coming.
-
Unbelievable, racism even exists in networking!
-
Beat me to it!
-
Those damn ones
-
-
It's funny aws report didn't mention 40% of aws sysops people were replaced by AI right prior https://blog.stackademic.com/aws-just-fired-40-of-its-devops-team-then-let-ai-take-their-jobs-d9db9d298bfa
-
this is unconfirmed and unlikely
-
"Leave a billion dollar company alone, leave it alone" Bro it's most likeliest thing ever
-
-
-
It was the best race anyone has ever seen 🫲🍊🫱
-
Let's be honest, not all races are equal
<br>
🫲🍊🫱-
Worst Race: Daytona 500.
Best Race: Kentucky Derby.
-
-
-
Just one more layer bro, just one more automated planning system bro and this time it will be entirely faultless please bro one more layer
-
I know a dude that talks like this... Like I hear his voice when I read this.
-
-
This is purely anecdotal, but I have been running into a lot of DNS issues over the past couple months where I work. 3 of the computers and even one of the laptops for remote work were having DNS issues that needed to be fixed. One even needed Windows reinstalled after fixing the DNS issue (Which was probably unrelated, but worth mentioning)
I'm honestly starting to think that the internet in general might be imploding. Not sure why, but replacing so many developers and programmers with AI might be responsible. Who knows, but it's definitely very strange.
-
The biggest issue is how centralized the internet has become. It went from a bunch of local servers to a handful of cloud providers.
We need to spread things out again
-
That's not how capitalism works though
-
But but Bezos has to pay for another rocket and yacht and he just got married!!!! Think about his quarterly statement! My god are you heartless!!!!!!!!
/s
(just in case it's not obvious)
-
-
A huge problem are developers who lack a fundamental understanding of how the internet even works. I've had to explain how short, unqualified names resolve vs how fqdns resolve. Or why even you may not be able to reach another node in your proverbial cluster, because they are on different subnets. Or, why using GUIDs as hostnames is a generally bad idea, and will cause things to fail in unpredictable ways, especially with deeply nested subdomains.
-
I have worked with too many devs that didn't even know what the 7 layers/OSI are or why they exist.
they didn't know what a network port was used for and why it's important to not expose 3306 to the internet.
they couldn't understand that fragmentation of a message bus occurs when you don't dedupe the contents.
you know, morons.
-
GUIDs?
Could you expand on that topic? :)
-
-
-
Ironically, my pihole is blocking that link. So here’s a clean one: https://www.theregister.com/2025/10/23/amazon_outage_postmortem/
-
Racist DNS!
-
I'm glad these things happen... it keeps everyone aware that cloud is fragile and Plan B should be considered for mission critical tasks.
I'm also hoping that it will improve cloud resiliency because a complete / partial restart of cloud systems needs a whole different approach than maintaining a running system.
-
Many different companies abruptly realized they need a DR plan for cloud outages
-
-
Removed by Moderator — Modlog
-
So, in the end they turned off the thing that caused this whole mess and everything is still working.
What's the point of having it, then?
-
They should check out YUNOhost.
-
Yeah, I don't get why they don't just put a RasPi in some corner, put PiHole on it and call it a day.
Geez, I mean, they could even charge extra for it, as they now block ads for their customers as well.
Like, imma gonna sell my advice to Amazon now, so they can clean up their act.
-
-
They got off sync.
So it is always DNS
can confirm, its always DNS. Even when it looks like a network issue, its DNS
Spotted the Network guy
https://isitdns.com/
I always view the source of websites like this and this is one of the worst I've seen. 217 lines of code (including inline Javascript?!) and a Google tag for some reason, all to put the word YES in green on black.
It’s always DNS