3w ago

Database Corruption Maintenance (00:01 - 06:00 UTC Dec 2)

stacks.programming.dev Database Corruption Maintenance

Programming.Dev is going to be going down starting at around 00:01 UTC Monday Dec 2 to attempt to fix the database corruption that happened relating to communities. This has been affecting attempting to look at the pages for certain communities (it would end up loading forever instead of actually sh...

Programming.Dev is going to be going down starting at around 00:01 UTC Monday Dec 2 to attempt to fix the database corruption that happened relating to communities.

This has been affecting attempting to look at the pages for certain communities (it would end up loading forever instead of actually showing the posts for the community) and was affecting federation in some cases.

This is the second attempt to fix the issues with the first one needing to be rollbacked from due to the fix not working.

This fix could take several hours, so the outage window is going to be from 00:01 - 06:00 Dec 2.

You're viewing a single thread.

19 comments

The fixes are done for tonight, but are not 100% complete. I estimate that 99.4% of cases have been fixed though. You might see problems with individual posts, but every community should be fixed. I've seen at least 1 issue with the !science@lemmy.world community and I'll be investigating that if it doesn't self resolve (stuff could still be catching up).
IF YOU SEE ANY MORE ISSUES PLEASE REPORT THEM HERE.
- https://programming.dev/c/programmer_humor, https://programming.dev/c/teach_programming, and https://programming.dev/c/data_structures still seem to be completely broken. Might need to investigate what's happening for those as well.
  CC: @Ategon@programming.dev
  
  Wait !programmer_humor@programming.dev is broken for you? For me in Jeroba that's one of the communities that's been working just fine this whole time, unless the recent maintenance broke it the last post I have is from yesterday.
  EDIT: Ok, so clicking your link, or even my link gives an error saying community can't be found but I can still pull it up from my subscriptions, so it was either broken by this maintenance or something weird is going on
  
  For me in Jeroba that’s one of the communities that’s been working just fine this whole time, unless the recent maintenance broke it the last post I have is from yesterday.
  That's strange indeed, the community has been inacessible from the web UI for weeks
  
  I can access many of the broken communities from external links and I might even be able to create posts in them from the UI but I cannot access their links directly from the UI, and they also don't load from my mobile App.
- Heyo, I assume this may also be related to the database issue, but I noticed today that posts I make to !linuxhardware@programming.dev from slrpnk.net aren't federating, nor are posts made there with my p.d account federating out to slrpnk.net.
- Edit: Definitely not relevant, it was a problem with the link I clicked
  
  You should use /c/ and not /post/:
  https://programming.dev/c/imaginarywitches@lemmy.dbzer0.com
  https://programming.dev/c/imaginaryfairies@lemmings.world
  
  Ah, you're right. I think I see what happened there, I hit those links from a sidebar in a community with href="imaginarywitches@lemmy.dbzer0.com"which breaks if you're on a post page instead of the community page. Easily fixed though, I'll ping the mods of that community.

19 comments

IF YOU SEE ANY MORE ISSUES PLEASE REPORT THEM HERE.