1y ago

Can Lemmy have distributed communities

I was thinking about this in regards to all the “defederation” posts.

Let’s say you spin up a server and over night it gets super popular and grows enormous. Now your yearly expenses shoot up and you’re forced to either look for a new host or shut down.

Now what if instead, you could get a few other people to spin up more small instances and distribute parts of your biggest communities to them, however the users don’t notice because The communities are looking across instances instead of within their home instance?

That’s the idea at least. This would allow for many things but most importantly, it would make things a bit more manageable. Thoughts?

15 comments

That's basically how it works already. When a user on server A is looking at a community on server B, they're not actually hitting server B. When the first person subscribes to the community, server A started downloading and caching all the content. The user is looking at it locally. Server A will then periodically synchronize with server B.
This means the primary load is on the server containing the users, not the server containing the community.

I do know that if you attempted this right now, all of the instances would need to access the same database somewhere which would retain a bottleneck.
Switching to a different distributed (eventually-consistent) database may be possible for Lemmy as a software but perhaps not possible (or difficult) for ActivityPub. I don't know that.
- I think the requests could just be routed for each "distributed community", but yeah, for the user data, a single database will have to be used, which is... not very secure to say the least.

I think it might be a better approach to have communities link together and if you post to one, it goes to all of them. However we should probably focus on making sure the basics work correctly first before dreaming big.

"Smart" or "Meta" communities List's like this: https://lemmy.intai.tech/post/2182
attached as metadata on a community with the option for users to subscribe to all related communities.
nitty gritty on assignment, data, etc to be fought in cage matches.
- This seems like a smart way to do it. If mods could make it an automatic one-click thing, and then let users combine those into a single merged feed, it would be a nice way to have a single cohesive community across multiple instances while still enjoying the benefits of federation.

You are talking about horizontal scaling. It's something which is non-trivial and would need the core of the app to be written in a very different way. Think of the challenges
Your sub-instance and your friend's sub-instance which serves a common community can be geographically far apart
They need to be synced
Internal function calls which were once local now need to travel across the network
Depending upon the heterogeneity of the hardware, there can be a large jitter (bad for UX)
The admin of my instance lemm.ee has tried to make horizontal scaling by decoupling some lemmy functions but pure horizontal scaling is really hard (glusterfs/spark).
- Would it be possible to have the dns of the instance technically be a load balancer and just point to different instances of the same docker container, but keep the database as a single shared entity?
  Edit: oh OP is talking about doing this on a sublemmy level instead of a site wide level. Would my idea work for the entire instance?
  
  dns of the instance technically be a load balancer and just point to different instances of the same docker container What you actually mean is the lemmy docker be loadbalanced. The DNS will only point to the loadbalancer.
  This is still somewhat doable and easier than what OP is asking for but I'm unsure whether lemmy can handle the synchronization issues. My instance manager (lemm.ee) has kind of distributed his architecture by decoupling core components of lemmy. Example
  DB on one server
  Loadbalancer on another server
  Main lemmy instance on one server

I don't think you can partition part of a sub across several instances right now. The best you can do is create an instance dedicated to one sub.
- I’m asking if it could be developed as a feature and if it would solve some of the issues we all see.
  
  That kind of feature is the kind of feature major internet companies spend hundreds of millions of dollars on each year. I think that there are better things for the devs to focus on.

You'd need a way to cluster the instance and the software doesn't provide that. You'd have to use some kind of 3rd party solution which would be more expensive than simply upgrading your server. Better to just co-op people into a server upgrade.

15 comments