8mo ago

Fediverse Apps on Kubernetes?

cross-posted from: https://lemmy.cloudhub.social/post/347779

I am running a Kubernetes cluster for this domain, and I'm looking at more services to run (right now I have Mastodon and Lemmy).
I was considering WriteFreely and PixelFed, but they don't seem to have an easy solution for running on Kubernetes (WriteFreely doesn't even have a production-ready docker image).
Is anyone else running federated services in their lab? Do you run any of them on Kubernetes?

25 comments

I run plenty of services in my lab on k3s. Most of them I had to manually build charts for and add them to my cluster. I doubt anyone has built charts for Lemmy, maybe mastodon. Anything that's dockerized like Lemmy can be put into kubernetes, but it's going to take some doing. Good luck out there, and of course if you get it working then I think the maintainers would be happy to get a helm chart merged into their repo.
- Yeah, that's the pain point - building and maintaining the charts.
  Also, I know the charts likely wouldn't have to be super complex, but I'm used to working with Bitnami's charts that are massively complex - I just don't have the time to go that in-depth.
  
  I start simple with mine. Start with the deployment, just getting the pod up and running, completely stateless. Get that stabilized, once that's happy then start moving onto the service, connecting to the UI. Then I finish anything else I can before adding in state - environment variables (first just in the deployment, then once that's done worry about a config map and secrets). (Adding volumes in just complicates it because then you have to reset those volumes if you want to go back to a clean state).
  As a list:
  Deployment (stateless) - get to not crash
  Service - connect to it
  Environment variables, extracting out what can be
  Secrets, adding those to the chart
  Volumes, adding in state
  Move to a nice clean setup with a clean values.yaml.
  Finally, once your happy, then make it conform to other standards. Getting it stood up in k8s is the hard part, then customizing can come after.
  
  I don’t like helm, so I use nix to maintain my fediverse deployments in kubernetes. Typically that'd just autoupdate itself to new releases, but for lemmy specifically I upgrade by hand nowadays since one release some time ago broke my deployment and its schema change was incompatible with the automated rollback.
  My setup is a combination of https://github.com/farcaller/nixdockertag (auto-updated docker imagesfor things where I fully own the deployments) and https://github.com/farcaller/nixhelm (for helm charts that I either consume verbatim PR have local patches on). Both just auto update nightly thanks to github.

If they’re containerized they’ll run in a Kubernetes cluster, I’m sure you know this. Why not build the K8s infrastructure you need to get them functional? Or maybe Helm charts if you’re up for it.
- Oh, I know I could get them to run with enough work. I just don't have that much time to spend on initial implementation and upkeep of the charts.
  I'm using FluxCD, which I believe can do deployments of plain Kubernetes manifests, but that still requires a decent amount of overhead to keep up to date.
  
  I just don’t have that much time to spend on initial implementation and upkeep
  Well k8s is a poor choice of platform for you :D

Using different federation protocol, but matrix wservers ould be the other big one.
Edit you also mentioned trouble creating them. I suggest looking into operator hub and using operators for postgres and redis and auth (keycloak?). This can get you down in the rabbit hole for making everything highly available too.
- Yeah, I used to host a Matrix instance - could do that for this one too.
  The issue is more about setting up the Kubernetes manifests and templating them. I usually use the chart's built-in postgres and redis config, though using an operator would make it more scalable for sure.
  I'm using Authentik for auth, but I do also like Keycloak.
  
  Yeah it's a bit of work sometimes. Synapse matrix kinda sucks too their philosophy of no environment variables for secrets. I ended up making an init container that hijacks my config map and I jet's the environment variables into the config

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters	More Letters
CSAM	Child Sexual Abuse Material
HA	Home Assistant automation software
~	High Availability
HTTP	Hypertext Transfer Protocol, the Web
k8s	Kubernetes container management package
nginx	Popular HTTP server

4 acronyms in this thread; the most compressed thread commented on today has 5 acronyms.

[Thread #740 for this sub, first seen 6th May 2024, 19:45] [FAQ] [Full list] [Contact] [Source code]

Found a couple, you should be able to find more on artifacthub.io
https://artifacthub.io/packages/helm/mastodon/mastodon
https://artifacthub.io/packages/helm/ananace-charts/lemmy

It's not kuberneties, but I run a family sized yunohost. It's great at installing and updating webapps. They have an awesome selection of federation apps like mastodon, writefreely, misskey, bookwyrm, and more.
For less than 5 users, I personally don't need kub, but it f I were to scale I would probably go that direction.
- I've seen that around, but I prefer to run my own services instead of relying on a ready-built system like that. I find they don't offer that much customization options usually.

25 comments