Skip Navigation

How much maintenance do you find your self-hosting involves?

I recognize this will vary depending on how much you self-host, so I'm curious about the range of experiences from the few self-hosted things to the many self-hosted things.

Also how might you compare it to other maintenance of your other online systems (e.g. personal computer/phone/etc.)?

71 comments
  • Very minimal. Mostly just run updates every now and then and fix what breaks which is relatively rare. The Docker stacks in particular are quite painless.

    Couple websites, Lemmy, Matrix, a whole email stack, DNS, IRC bouncer, NextCloud, WireGuard, Jitsi, a Minecraft server and I believe that's about it?

    I'm a DevOps engineer at work, managing 2k+ VMs that I can more than keep up with. I'd say it varies more with experience and how it's set up than how much you manage. When you use Ansible and Terraform and Kubernetes, the count of servers and services isn't really important. One, five, ten, a thousand servers, it matters very little since you just run Ansible on them and 5 minutes later it's all up and running. I don't use that for my own servers out of laziness but still, I set most of that stuff 10 years ago and it's still happily humming along just fine.

  • Typically, very little. I have ~40 containers in my Docker stack and by in large it just works. I upgrade stuff here and there as needed. I am getting ready to do a hardware refresh but again with Docker that's pretty painless.

    Most of the time spent in my lab is trying out new things. I'll find a new something that looks cool and go down the rabbit hole with it for a while. Then back to the status quo.

  • For some reason my DNS tends to break the most. I have to reinstall my Pi-hole semi-regularly.

    NixOS plus Docker is my preferred setup for hosting applications. Sometime it is a pain to get running but once it does it tends to run. If a container doesn't work, restart it. If the OS doesn't work, roll it back.

  • Not heaps, although I should probably do more than I do. Generally speaking, on Saturday mornings:

    • Between 2am-4am, Watchtower on all my docker hosts pulls updated images for my containers, and notifies me via Slack then, over coffee when I get up:
      • For containers I don't care about, Watchtower auto-updates them as well, at which point I simply check the service is running and purge the old images
      • For mission-critical containers (Pi-hole, Home Assistant, etc), I manually update the containers and verify functionality, before purging old images
    • I then check for updates on my OPNsense firewall, and do a controlled update if required (needs me to jump onto a specific wireless SSID to be able to do so)
    • Finally, my two internet-facing hosts (Nginx reverse proxy and Wireguard VPN server) auto-update their OS and packages using unattended-upgrades, so I test inbound functionality on those

    What I still want to do is develop some Ansible playbooks to deploy unattended-upgrades across my fleet (~40ish Debian/docker LXCs). I fear I have some tech debt growing on those hosts, but have fallen into the convenient trap of knowing my internet-facing gear is the always up to date, and I can be lazy about the rest.

  • Minimal, I have to force myself to check the servers for updates atleast once a week.

    Main problem for me is I automated podman and docker updates with their respective autoupdate mechanisms and use ntfy for push notifications so I know if a service stops working and I had an update recently on it that it's an update issue.

    Also have uptime monitor wih uptime kuma to monitor state of my services to catch them not working before I do, also ntfy for push notifications.

    Also have grafana+prometheus seted up on my biggest server for monitoring and alerting with alertmanager+mail to get notifications on even more errors.

    So in general I only have to worry about occasional once every few months error and updates of the host system (debian).

  • As a complete noob trying to make A TrueNAS server, none and then suddenly lots when idk how to fix something that broke

  • I run two local physical servers, one production and one dev (and a third prod2 kept in case of a prod1 failure), and two remote production/backup servers all running Proxmox, and two VPSs. Most apps are dockerised inside LXC containers (on Proxmox) or just docker on Ubuntu (VPSs). Each of the three locations runs a Synology NAS in addition to the server.

    Backups run automatically, and I manually run apt updates on everything each weekend with a single ansible playbook. Every host runs a little golang program that exposes the memory and disk use percent as a JSON endpoint, and I use two instances of Uptime Kuma (one local, and one on fly.io) to monitor all of those with keywords.

    So -

    • weekly: 10 minutes to run the update playbook, and I usually ssh into the VPS's, have a look at the Fail2Ban stats and reboot them if needed. I also look at each of the Proxmox GUIs to check the backs have been working as expected.
    • Monthly: stop the local prod machine and switch to the prod2 machine (from backups) for a few days. Probably 30 minutes each way, most of it waiting for backups.
    • From time to time (if I hear of a security update), but generally every three months: Look through my container versions and see if I want to update them. They're on docker compose so the steps are just backup the LXC, docker down, pull, up - probs 5 minutes per container.
    • Yearly: consider if I need to do operating systems - eg to Proxmox 8, or a new Debian or Ubuntu LTS
    • Yearly: visit the remotes and have a proper check/clean up/updates
  • I have just been round my small setup and run an OS update, took about an hour. That includes a reboot of a dedicated server with OVH.

    a pi and mini PC at home, a dedi at OVH running 2 LXC and 5 qemu vms. All deb a mix of 11 and 12.

    I spend Wednesday evenings checking what updates need installing, I get an email every week from newreleases.io with software updates and run Semaphore to check on OS updates.

  • Not much for myself, like many others. But my backups are manual. I have an external drive I backup to and unplug as I intentionally want to keep it completely isolated from the network in case of a breach. Because of that, maybe 10 minutes a week? Running gentoo with tons of scripts and docker containers that I have automatically updating. The only time I need to intervene the updates is when my script sends me a push notification of an eselect news item (like a major upcoming update) or kernel update.

    I also use a custom monitoring software I wrote that ties into a MySQL db that's connected to with grafana for general software, network alerts (new devices connecting to network, suspicious DNS requests, suspicious ports, suspicious countries being reached out to like china, etc) or hardware failures (like a raid drive failing).... So yeah, automate if you know how to script or program, and you'll be pretty much worry free most of the time.

  • My mini-pc with Debian runs RunTipi 24/7 with Navidrome, Jellyfin and Tailscale. Once every 2-3 weeks I plug in the monitor to run updates and add/remove some media.

  • Maybe 1-2 hours a week for ~23 docker containers, 3 LXCs and proxmox, so not much. Most of that time is spend SSH-ing doing minor updates. Running Debian on everything has been amazing. Stability is just phenomenal.

  • For my local media server? Practically none. Maybe restart the system once a month if it starts getting slow. Clear the cache, etc.

    When I hosted game servers: Depending on the game, you may have to fix something every few hours. Arma 3 is, by far, the worst. Which really sucks because the games can last really long, and it can be annoying to save and load with the GM tool thing.

    • When I hosted game servers: Depending on the game, you may have to fix something every few hours. Arma 3 is, by far, the worst. Which really sucks because the games can last really long, and it can be annoying to save and load with the GM tool thing.

      Was that a mix of games being more involved and the way their server software was set up, from what you could tell, or...?

      • A bit of both. It really depends on the game. Some games are super simple, just launch an executable and hand out the IP. Others are needlessly complicated or just horribly coded. My example game is just an absolute mess all around even just as a player; running a server is no different. And since the actual game is all user-made, sometimes the problem is the server software, and sometimes it's how the mission you're running was coded. Sometimes it's both.

  • I just did a big upgrade to my "home lab" (got a new switch and moved it out of my bedroom), which required some maintenance in the days after the upgrade... Running a new ethernet cable, because the old one just couldn't heck doing gigabit, reconfiguring my router and AP, just general stuff like that.

    Other than that and my DHCP/DNS VM sometimes forgetting to autostart after a power outage, pretty much 0 maintenance

  • i've got a RPI and other SBC, once month, make a copy of the MicroSD card, as the data is in the HD

  • Sometimes its real easy and I‘m taking a month off and nothing breaks. Then I have times where I want to add new services or optimize stuff. This can take forever. Right now I‘m building object storage behind a vpn.

  • Depends what are you doing. Something like keep base os patched is pretty much nil efforts. Some apps more problematic than others. Home Assistant is always a pain to upgrade and something like postfix is requires nearly 0 maintenance.

  • @ALostInquirer@lemm.ee Not much tbh, I host email, a git server, activitypub, change detector, healthchecks, libreddit and another dozen of services in 3 different servers.

    Every now and then I check manually the backups, because it is the sane thing to do. Also I try some new services on docker, but that is less and less common tbh.

  • That must be why it stopped working ;-)

    Does 48 hours not getting a reverse proxy working count?

    It’s FreeNAS and I don’t really hoast anything but the plex server… so 48 hours.

    If deleting files counts 10 days a year, if not 1 day a year.

71 comments