2y ago

Nextcloud on VM or Docker? Best set-up for proper backups?

Hi all, sorry if this has been asked/discussed before (I couldn't find any directly overlapping posts):

I have been running the Nextcloud snap now for quite some time, and although things have run quite smoothly, I never really managed to properly back things up.

I make weekly backups of the database, config and data, but it's very hard and time consuming to glue these elements back together. And as they say: when you can't check whether a backup works, it's not really a backup.

I have been experimenting with KVM/qemu lately and things look pretty great. The idea of simply backing up the entire OS that runs Nextcloud (a backup that you can easily deploy/run somewhere else to test if it's working) sounds very attractive.

Reading around, however, tells me that some of you recommend running the Nextcloud docker (instead of a VM).

My questions:

What would be the advantage of running Nextcloud as a docker, instead of within a VM?
What would be a sensible way to have an incremental/differential backup of the VM/Docker?
The storage usage of my Nextcloud instance exceeds 1TB. If I run it within a VM, I will have to connect it to a 2TB SSD. Does it make sense to add the external storage space to the VM? How does that affect the ease of backing the full VM up? Or (as I have read here and there) should I simply put the entire VM on the external SSD?

22 comments

Docker is a powerful tool to increase confidence in your backups.
In a VM, the way you figure out which files to backup is to read the docs. If they're wrong or you misread them, the only way you'll find out is by doing a full restore test... which is often painful and complex in home setups.
In docker, the filesystem outside volumes is destroyed between every container restart. If your volume setup is insufficient, you'll repeatedly lose state during your initial installation process between container restarts. You'll continually test your state management throughout the lifetime of the service during restarts. This leaves a much smaller window for backup mistakes.
The tradeoff with docker is that the networking is complex (well, everything is complex... but the networking is where it often hurts). But if you're able to deal with that one-time pain, it's superior almost all the time for home setups. I think the only things I run outside docker are ssh and netdata. SSH because it's stateless and works perfectly out of the box, and netdata because it wants permissions to everything... and is functionally stateless for me because I don't care if I drop my observability data.

From my experience docker seems to be best for me. I’m also no expert in any of this.
What I do is run the container in docker and then I user rsync to backup my files to both a secondary hard drive and off site storage with a backup provider.
I haven’t looked into database backups yet. Just files.
- docker exec nextcloud-mariadb-1 /usr/bin/mariadb-dump --defaults-extra-file=/backup/.mylogin.cnf -u root --single-transaction --quick --all-databases |gzip > /mnt/mysql/backup/nc${NUM}_dump.gz
  You are welcome
  
  What would be the difference between this and creating a volume to the data directory and backing up that one?
  I mean, is it not recommended? Is there a potential to backup in the middle of a DB transaction which would make the backup useless?

I'm not really an expert when it comes to backups, so I'll only answer point 1:
The general advantage of docker over VMs is that it is not a full virtualization. A docker container still shares the same kernel as the host system (at least on Linux); thus processes in a container essentially run on your host and you can see them in your task manager (they are just heavily isolated). This also means it's way easier to connect the filesystem (see https://github.com/nextcloud/docker for details). This generally makes docker containers more performant than full VMs. VMs also need a static allocation of memory, where docker simply shares the host memory and uses as much or little as it needs.
Of course this also means that a process can more easily gather information on your host, since it's not full virtualization.
Like with VMs, you can easily just start the container(s) again, and you only need to save the data, since the container can be rebuilt.
I'll add that the guide linked above uses docker volumes, which are docker-managed spaces for your data. You can export and import a volume, but generally don't have access to the data (afaik). You can also mount a folder on your filesystem instead if you prefer to have acces all the time.
- Yes, indeed. One of the nice things about docker is that you can keep everything self contained, but then also map in volumes. This may be an external directory for configuration that you archive elsewhere but could also be something more advanced like a Kubernetes PVC.

Im running nextcloud in docker from lsio atm. Doing daily file backups, so not database, mostly because it seems its pain to test backups, but also dont rly need anything else (at least not for now).
There is also a community !nextcloud@lemmy.world
First time I hear about running nextcloud in VM. Following to learn more.

I have been FIGHTING TOOTH AND NAIL for about a week to get the AIO working in docker on Linux, and I'm getting extremely frustrated with it.
I FINALLY got it to actually function yesterday to where I could attempt to do its internal setup, but now I'm stuck with this page:
I'm genuinely starting to question whether it's worth it at this point, I haven't once been able to actually get it all the way set up and functional.
- I tried the AIO image as well, I would recommend against it. https://github.com/nextcloud/docker is a more manual setup, but it's also much more flexible. AIO forces you to have a domain name and HTTPS certificate etc, which might not be necessary for you.
  As for the page you are seeing, this is the administration page afaik, the actual nextcloud interface is running on a different port (https 443 with AIO).
  
  I just keep hitting issues with the damn AIO, I got past this and now it's stuck in maintenance mode. Who the fuck thought this was in a release ready state? I swear I've never had this much issue with ANY other docker container- and the documentation doesn't help at all. I'm at a loss here, i'm super frustrated with this.

I have the snap installed, for what it's worth it's pretty painless AS LONG AS YOU DON'T WANT TO DO ANYTHING SILLY
I've found it nearly impossible to alter the base behaviour and have it not entirely break, so if nextcloud out of the box does exactly what you want, go ahead and install it via snap...
I predict that on docker you're going to have a bad time if you can't give it host network mode and try to just forward ports
That said, docker >>>> VM in my books

Follow the official documentation https://docs.nextcloud.com/server/latest/admin_manual/installation/
Plain old webserver + php-fpm + database engine (I use postgres), no docker. People not following the recommended setup seem to have all kinds of woes (https://lemmy.world/comment/1981161)
- Don't they offer an official all in one docker image?
  https://github.com/nextcloud/all-in-one
  
  It's more of a pain to get running than you might think for an official image. I got it running but definitely had issues getting started.
  
  I have been trying to get this working for over a week now and have run into LITERALLY EVERY POSSIBLE PROBLEM. It's driving me nuts and I'm on the verge of giving up. Do you happen to be any good with the AIO image setup?

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
Fewer Letters More Letters

HTTP Hypertext Transfer Protocol, the Web
HTTPS HTTP over SSL
SSH Secure Shell for remote terminal access
SSL Secure Sockets Layer, for transparent encryption
2 acronyms in this thread; the most compressed thread commented on today has 7 acronyms.
[Thread #34 for this sub, first seen 13th Aug 2023, 08:55] [FAQ] [Full list] [Contact] [Source code]

Fewer Letters	More Letters
HTTP	Hypertext Transfer Protocol, the Web
HTTPS	HTTP over SSL
SSH	Secure Shell for remote terminal access
SSL	Secure Sockets Layer, for transparent encryption

What would be the advantage of running Nextcloud as a docker, instead of within a VM?
No idea really beyond the usual VM/container trade-offs, I guess it would allow you to use orchestration tools and similar for Docker.
What would be a sensible way to have an incremental/differential backup of the VM/Docker?
If you use Proxmox as your hypervisor it comes with a sophisticated backup solution, probably the same for ESXi or whatever. Not sure about Docker.
The storage usage of my Nextcloud instance exceeds 1TB. If I run it within a VM, I will have to connect it to a 2TB SSD. Does it make sense to add the external storage space to the VM? [...]
That's what I would do at least. Connecting external storage space to a VM/container is relatively trivial and Nextcloud recommends to separate binaries and data directory anyway. Plus this allows you to use different backup strategies for data versus binaries+metadata.
In case you haven't yet, I'd also recommend taking a look at this: https://github.com/nextcloud/vm
It's basically a collection of three shell scripts to install, manage, and update Nextcloud. Last time I tried it also worked on LXC/LXD, not only VMs. It would probably work on Docker as well and has some files related to that in the migrate/docker directory.

@notsofunnycomment Best set-up would be no Nextcloud at all. If you only use it for files, take a look at OCIS, SyncThing or Seafile.

My friend and I run most things in kubernetes (k3s), and then we use longhorn to backup volumes, which then can be re-used if your cluster crashes. Here's a blog post describing the process (although not for nextcloud specifically, it could be applied to Nextcloud as well, as we do this for nextcloud as well)
Octopusx blog: Backup and Restore

If you can use containers always use containers as a rule of thumb. VMs are less efficient in almost every way and they add some unnecessary complexity.
For docker you basically only have to backup the persistent data. So in case of the docker setup you just have to backup the mounts and probably your compose file you are using. This probably also answers your third question already. Container files can be left alone and don't need to be considered for backups as they should be stateless and can reside in their default location (/var/lib/docker/overlay2 or so by default).
Overall it is quite simple as you only really have to consider the mounts and the docker setup. The mounts you define and should be really obvious and the docker setup is just a few config files at most or just the compose file.

22 comments