Skip Navigation
..:: tchncs ::.. @discuss.tchncs.de
Milan @discuss.tchncs.de
Matrix server

[Matrix server] Upcoming maintenance and backup restore July 31

Well hello again, I have just learned that the host that recently had both nvme drives fail upon drive replacement, now has new problems: the filesystem report permanent data errors affecting the database of both, Matrix server and Telegram bridge.

I have just rented a new machine and am about to restore the database snapshot of the 26. of july, just in case. All the troubleshooting the recent days was very exhausting, however, i will try to do or at least prepare this within the upcoming hours.

Update

After a rescan the errors have gone away, however the drives logged errors too. It's now the question as to whether the data integrety should be trusted.

Status august 1st

Well ... good question... optimizations have been made last night, the restore was successful and ... we are back to debugging outgoing federation :(

The new hardware also will be a bit more powerful... and yes, i have not forgotten that i wanted to update that database. It's just that i was busy debugging federation problems.

References

20 comments

  • Thank you so much for all the work. I have tried on many occasions to create an account in matrix these last days and it has been impossible for me. I never get the email verification.

    Again, thanks for all the work you do.

  • Thanks for putting in the work. Is there anything we can help you with? From what I understood the domain is german, is the server in germany as well? I‘m located in germany and do sysadmin work. Fighting with hosting companies is part of my job. ;) let me know if I can do anything. Have a good one!

    • Thank you :) Well i am not sure if there was something to fight over except maybe some sort of refund... for now it seems to be fine one the new machine. – yes, i am from germany, however i think its a helsinki dc from hetzner.

      • You’re very welcome. Hetzner is generally a good host afaik. It does depend on the configuration I suppose. Are you using the shared vps or something else? If the storage is guaranteed (as in not custom hardware) they are technically responsible for its condition. A host I‘m working with (also located at hetzner but in falkenstein) does 2 backups a day which also prevents having to revert far back.

20 comments