1y ago

PeerTube has a federation problem

With the mass migrations of Reddit users to Lemmy/Kbin, and Twitter now speedrunning its own mass extinction, it seems me that the eventual future of social media is de-centralized. I like how Lemmy is slowing turning out, even if it still has some work to do and growing pains to fix up. It's still able to inform me of all of the current events I want and has a large enough community that it doesn't feel empty.

I think a similar path will present itself for a de-centralized video media platform like PeerTube, since YouTube will eventually piss off enough of its users to cause a similar kind of exodus. Wanting to jump in on the concept at an early stage, I signed up for a channel on spectra.video and uploaded my video collection there.

But, I don't really see the same kind of community and usefulness on PeerTube. I check out the Discover and Trending pages, and it just seems like the same set of videos, really. There's not enough content to keep PeerTube from looking like a small indie project. I can click on Recently Added and it is usually other people just dumping their channel collections, instead of recent adds of new videos. It's very easy to scroll down and find videos from months ago.

After poking around on various other PeerTube sites, I think I found the real problem with the platform: Federation.

For example, let's look at how federated Lemmy's community is:

All interconnected with hundreds and hundreds (if not thousands) of other instances. If you sign up for one Lemmy account, you have little risk in not being able to access a remote community elsewhere. It feels like a federated community, where everything is de-centralized, but communication is linked everywhere. I can even link to my own video channel from Lemmy.

Now, look at PeerTube's instance lists, based on what I've seen on the Join PeerTube site:

It's all so bare. At most, 80-90 instances for some sites. I can't see a lot of other instances' videos, and they can't see mine. Not from here or here or here or here or here or here or here or here.

It makes PeerTube a large collection of small silos, instead of a real federated community. People want to be able to sign on to an instance and find the content they want without having to jump through all of these different instances. Subscription feeds rely on having a unified list from many different instances. The technology has a lot of potential, but the PeerTube community is not nearly as organized as the rest of the Fediverse.

This sounds like a somewhat simple problem to solve, but I'm not sure what other kind of technological hurdles exist. How did the Lemmy community solve it?

71 comments

I dunno. I think trying to treat peertube the same as lemmy is going to be impossible. Video hosting and sharing is a massive data hog. It's going to take a dedicated non-profit organization to make it viable. Without a backbone like mastodon has, I don't see it ever being anything useful.
- Not to mention monetization. A big reason Youtube is what it is is because it pays its creators.
  
  YouTube pays its creators in peanuts. That's why almost every YouTube video has a sponsor, or is thanking its Patreons, or both.
  I have a full-paying job, so I don't bother with monetization. I feel like monetization is a boat anchor designed to shackle creators with arcane unspoken rules and unfair copyright claims. (My Babylon 5 video is still technically marked as ineligible because I criticized TNT when talking about Crusade.) I specifically signed up for Google AdSense to turn off ads on my YouTube videos.
  I think what PeerTube is doing by having a Support section is good enough. Donate to your instance or donate to your creators directly. It's a helluva lot more money than YouTube would be paying.
  
  True that!
- Yeah, I dont think it's a problem of lack of interest but one of resources. You literally need in the 100s of terabytes of storage space, especially for HD videos.
  
  Even if fully decentralized. Many vps providers and isp have bandwidth limits. One video on your self hosted channel goes viral and boom. Instance is down.
- YouTube hasn't had the same meltdown as Twitter and Reddit, which might explain PeerTube's relatively small scale.
  The meltdowns I mentioned brought a lot of attention to their Fediverse alternatives, but YT hasn't really had such a meltdown because Google is somewhat smart and knows not to rock the boat too much lest they have a Twitter or Reddit moment.
  
  Google (YouTube) is banning users using AdBlock. They're also working on installing internet-wide DRM. So yeah, they're having the same "meltdown".
- I think there needs to be an option to link to a PeerTube instance without having to always host the material. Have some sites where you host videos, but other peers where you don't because you can't afford to host all of it. That would at least solve the federation problem. Hopefully, they can have enough "tier 1" peers (seeders) to serve the content from multiple sources.
  Also, why can't videos use torrent technology to serve the data by other viewers?
  
  Also, why can't videos use torrent technology to serve the data by other viewers?
  from what i read, they do.
  i think the problem is the kind of distributed systems it was designed to be vs the kind we are talking about.
  PT won't be, at the current state of propagation of content, redundancy and accessibility of content, a replacement for YT or similar.
  it could be, but today it cannot.
  there is another post mentioning how if certain Lemmy instances keep growing they'll need economic support.
  it smells PT, the way it's organised today, needs support right now.
  there are various ways to optimise data propagation and replication for scalability of content.
  I think the discussion needs to be open towards this question
  
  Yeah, I think this might be the best for PeerTube. Instead of federating to the point of copying the actual videos to each server, just copy the metadata/thumbnails. That way all of the videos on all federated servers on the network show in each instance's list of videos, but when you actually click on a video it requests an embedded player from the original server that the video was hosted on rather than on your local instance.
- YouTube should be a part of the public infrastructure.

Hi, so I run Spectra, and would like to weigh in.
I wrote this piece a few years back about the content situation on PeerTube.
TL;DR, PeerTube has a significant problem with spam. I'm not just talking about spammy comments, although it has those, too. No, I'm talking about the videos. For example: there's an option within PeerTube that, when enabled, basically just automatically subscribes to every server that subscribes to yours. I let that setting run for a while, and connected to maybe a hundred random instances over time.
It was all garbage. Either you get far-right propaganda videos, actual nazi videos, or super random weird stuff of little value. Want a video in Hindi for a restaurant with a two-second video featuring a French TV commercial transcribed from VHS? How about that, mixed with thousands of random snippets of media that you will never care about or relate to?
A lot of PeerTube admins kind of informally got together and said: you know what, this is crap, no one is ever going to enjoy this. So, we connected our communities together. We have to do our research on which servers are good, and which ones just serve up bullshit. Good community stewardship, in this case, requires us to do our homework on which servers are worth following. Instead of following as many servers as possible, we're more inclined to check and see if the place is putting out original stuff, has decent guidelines, and isn't spouting hateful crap everywhere. To build community organically, we have to do so with intention.
The reason that you're not seeing your videos in any of the places you've listed is because their servers don't follow ours. This doesn't mean that your videos cannot be seen through federation - it's just that, in any of those places, no one is subscribed to you, and that server isn't subscribed to our server. So, your channel and videos aren't likely to show up there, unless somebody actively chooses to subscribe to you.
I agree that PeerTube is seriously lacking in some kind of Community Discovery feature, and would be greatly enhanced by it.
- Hey, Sean, thanks for inviting me to your community and replying to this post. I read through your blog article, and I'm glad we seem to be aligned with our goals here.
  I'm still curious why the situation seems so different between the Lemmy and PeerTube instances. I'm not sure if the Lemmy admins are doing the same level of curation, but given the sheer amount of instances they connect to, I doubt it. So, I think they are using the same automated subscription feature you tried using. Yet, the content of even unfiltered new posts on any Lemmy instance are fairly high quality.
  Comparing the stats between Lemmy and PeerTube has rather strange differences. I would have expected PeerTube to be still fairly low population, compared to Lemmy, with its recent Reddit migrations. But, no, PeerTube actually has a comparable user count to Lemmy. Other observations:
  PeerTube has 15x more posts than Lemmy, which contributes to the video quality problem
  The top list of servers on Lemmy are mostly the ones you'd expect, and are the ones interconnected with each other. The top list of PeerTube servers are... not. Like, truly some WTF ones in there.
  As a French-developed app, PeerTube's top servers are mostly non-English. Given the obvious language barrier, that makes it difficult to interconnect without much better language filters.
  Lemmy has user voting and per-channel moderation. PeerTube has likes/dislikes, but it's not immediately visible or usable, and the channel is owned by the same person uploading videos, so it's not really moderated in the same way.
  So, I guess the approach you're taking seems to work with the tools you have available. But, I also hope the development team continues to hammer at this problem, because the PeerTube communities seem to be much more fractured.
  
  On that, we agree. While what we have been Spectra and Diode and the handful between us is...fairly decent? Discovery still absolutely sucks.
  I think one thing that works well for Lemmy is in how its communities are structured. Like this one! A bunch of servers all connect to the same space, and people passing through trade thoughts, questions, and bits of news.
  I feel like part of the problem is that PeerTube has no such communal structure. You just kind of... stumble around and try to watch videos and hope it's interesting. In fact, in the past, people shared their videos through Reddit communities like /r/PeerTubeVideos. It's like we had to bootstrap it with something else.
  Maybe we should do the same with Lemmy in the interim?

I think you're missing the forest for the trees here.
PeerTube accounts are for video makers, not video watchers. If I want to watch a PT video, I can do so without an account. If I want to leave a like or comment, I'll do so from my mastodon or misskey account. The reason Peertube servers don't tend to federate with each other is because they don't need to: they federate with mastodon/misskey/pleroma servers for the people who want to watch. (also, the https://spectra.video/about/follows page OP linked seems to ignore non-peertube follows?)
As for finding new videos on PeerTube, I recommend using sepiasearch.org
- PeerTube accounts are for video makers, not video watchers.
  Personally, that seems like a short-sighted way to use the platform. If there was ever an exodus of YouTube users, new users should be able to sign up to a PeerTube instance and browse in a similar manner as YouTube. They would search for "PeerTube", find the "join PeerTube" web site, and sign up, just like how they joined Lemmy or Mastodon. Most wouldn't think to log into a local Lemmy or Mastodon instance and try to somehow link to a completely different platform like that.
  If I want to watch a PT video, I can do so without an account.
  People will, very quickly, want to be able to subscribe to channels.
  If I want to leave a like or comment, I’ll do so from my mastodon or misskey account.
  Nothing about this interface tells me I can log into an outside account. In fact, if I happen to click a link to watch a video, there's nowhere to go to in order to watch the video from my non-PeerTube account. I would have to log into my local Lemmy instance, and find the video from the POV of the local instance, like here.
  Even then, I can't just watch the video and write a comment at the same time, in any sane way. I would have to watch the video in a different tab, from the PeerTube web site, and then go back to the local Lemmy instance to make my comment. And the PeerTube tab would take a while before the comment shows up.
  To a user that is either non-technical or not familiar with how ActivityPub protocols work, this is a very jarring experience, if they even get that far. Most would give up in frustration far far sooner.
  The reason Peertube servers don’t tend to federate with each other is because they don’t need to: they federate with mastodon/misskey/pleroma servers for the people who want to watch.
  None of these have YouTube-like front-ends to discover new videos or look at a subscription feed. The PeerTube page does.
  Video is video. Posts are posts. Not everybody likes mixing the two.
  I watch videos from my TV far more often than I watch from a computer or phone, and I'm sure other people do, too. So, we'll eventually get to the point of a PeerTube TV app on Roku or Android, which would need to have the ability to log into a PeerTube instance to access their subscription feed.
  As for finding new videos on PeerTube, I recommend using sepiasearch.org
  That's neat, but it's missing a good front-page view like this.
  
  People will, very quickly, want to be able to subscribe to channels.
  Mastodon can do that. On mastodon channels are groups that boost posts, which means they can be followed. Mastodon handles lemmy similarly.
  Even then, I can’t just watch the video and write a comment at the same time, in any sane way.
  Yes, but that’s even hard on youtube, because you have to scroll down. It’s easy to do though, despite quite hidden. If you aren’t logged in and press comment or subscribe peertube asks you for you fediverse handle and redirects you to your fediverse instance (on mastodon: corresponding post for comment; follow popup with the group/user for subscribe). In that regard it works a lot better than lemmy.
  In fact, if I happen to click a link to watch a video, there’s nowhere to go to in order to watch the video from my non-PeerTube account.
  Which gets us back to this, because mastodon embeds the video into the post (the webinterface at least). Clicking on comment does exactly this, but it’s definitely not intuitive and it’s still the player from that peertube instance. I don’t know if it works with anything other than mastodon. It certainly doesn’t work with lemmy right now.
  Here a screenshot of it, because I can’t give a link. Mastodon will immediately redirect you to peertube if you aren’t logged in.
- I think that's the point, I used peertube for video sharing along with mastodon post for reach. I guess there is not a way to link mastodon snd peertube accounts, but that's not unheard of also in non-federated social media.

My opinions are the same as yours, in my little experience. There’s not a lot of content on there, you can’t see content from other instances, and the content that is on there is not high quality.
One thought that crossed my mind was having a PeerTube instance backing each of the major Lemmy instances for video upload. Anything to reduce our dependence on corporate frequently used sites like YouTube and imgr.
Ok so here’s where it starts falling apart in my head. First, as another poster mentioned, the costs would be through the roof compared to a link aggregating site like Lemmy. From what I understand PeerTube does some Bit torrenty stuff to reduce bandwidth usage. And IPFS could help as well. But at the end of the day you need a server hosting this as a source of truth, with the monumental cost that accounts for.
The other huge problem is moderation. We need strong moderation because jackasses are going to upload CSAM. As single files. Spliced into the middle of legitimate videos. And the fediverse is way too important to have it be associated with that crap. So like I said, extremely strong moderation, for free.
I want to see Peertube take off and overcome these hurdles. If there’s anything I can help develop I’d be happy to take on a ticket.

I think you're totally right. The big difference is that while for a lemmy instance to show you other instances content under 'All' only one other user from your instance needs to subscribe to it and you will be exposed to it. On PeerTube if another user subscribes you will never know about that instance because the server admin needs to manually federate with that instance.
But because there is no good differentiation between 'All' and 'Local' many admins - including me - just don't federate with other instances automatically. What made it worse - I'm not sure if it got better over time - was that if you automatically federated then there would be naked people without NSFW at the top of every discovery, there would be antivax, nazis, all the fringe groups which weren't allowed on YouTube anymore. That would make it impossible to find the local videos and your instance would look like it's run by nazis and would offer bad quality porn.
- I'm not sure I understand the difference between those problems on PeerTube vs. Lemmy. Lemmy NSFW instances exist, especially since the Reddit NSFW community established a foothold, and they are all properly separated into their own groups and instances. Lemmy admins maintain their own block lists (see the bottom of the instance lists linked) to get rid of the antivax/nazi nonsense, which usually isn't that large, compared to the thousand or so instances they connect to.
  Heck, even Lemmy.film and others have links to the porn/NSFW instances, so somehow they are maintaining a relationship to keep things clean on the non-NSFW side.
  
  This is just my observation, but I think it's because the communities on PeerTube decided to just give up on controlling moderation of instances (perhaps because the instances were popping up like whack a mole?)
  So they opted instead to just give up on the idea of any federation on the PeerTube branch of fedi, and now we have the isolated issue of today...

I've been so frustrated by Peertube because I feel like it needs leadership that the developers just aren't willing to do. It feels like someone made a product and then never actually used it.
- I was just thinking the other day if the dev is even running an instance and I almost certain that they don't.
  
  @jeena well, they do, but it's not open to registrations: https://framatube.org/
  @sabreW4K3

I still don't understand the content that you can find on there. Can I find commentary videos from my favorite YouTubers on there? Music videos? Music theory videos? Meme videos? Is it all only educational stuff or technojargon coding nonsense?
- You won't find youtube content on there. Like with reddit and lemmy, creators and users have to switch to bolster engagement.
  
  Okay, but what channels are there to watch?

I observed the same segmentized silos problem a little while ago. Initial reach is SOOOO much harder when federation doesn't work.
Reach is there only when some popular tech vblogger appears, like The Linux Experiment on TILvids - and that's it. Most people are there for that channel and that channel only (Nick's done a good job at promoting it, as we see), which must mean the impression other parts leave on a viewer is considerably worse.
- TILvids has the worst possible solution by subscribing to nobody in the PeerTube landscape.
  
  It’s correct they don’t federate with anyone else directly, but it is possible for another instance to follow channels on tilvids and have them show up as viewable content on the other instance.
  
  TILvids is like your average overprotective uncle...
  
  It would be a good solution if peertube was growing at a decent pace. But it's not so it ends up just dragging it down

Hosting & recoding video is very expensive. Hosting social media postings is relatively inexpensive.
I think it's as simple as that. Not many people can afford to host peertube instances.

Most instances do not enable global index to prevent having to do much federation between PeerTube instances.

In my mind, having a peertube account is only for the video makers, as a consumer of video you are better off interacting with it from another app (lemmy is so-so about this, but mastodon/misskey/firefish seems to work better) or following the RSS feed
- Which then again is too "techy" for the average person. We both are less likely to be the average person, see.

What about https://d.tube/ ? At first glance, you may think Dtube is just another of those websites similar to YouTube, but that’d be doing the platform a disservice.
Dtube is a decentralized platform that uses a social blockchain system where every user interaction is end-to-end encrypted. There’s no central figure controlling what users are posting, so they have more independence and control over their content. As a result, Dtube is censorship-free, and every uploaded video remains there forever.
The best part of this innovative video website is that users earn rewards for posting, curating, and voting content. The rewards come in the form of Dtube’s own cryptocurrency, DTC.
video websites like youtube
- Blockchain 🤮
- @LoveSausage @MetaStatistical
  I just tried it but I recognized that all videos I watched are hosted and played by YouTube. It just presents the regular YouTube player inside the drive website.... did I get something wrong here?
  
  https://www.reddit.com/r/dtube/comments/cyhxd7/why_are_dtube_videos_mostly_just_embedded_youtube/ The reason we allow people to use third party services is that for many, the concept of self-hosting and getting away from abusive services isn't a problem. They'd simply like a video site that allowed them to earn DTC and Steem.
  I'd agree that having ONLY youtube videos would defeat the purpose of a decentralised video service, but we allow a lot of third party services (as long as they use oembed, you're good to go). Alternatively you can let us host the video, finally if you have the know how and a local IPFS node, you can self host.
  The point is we provide a lot of options, and that's the key to decentralisation, not that everyone does it, just that everyone has the ability to do what they want to.
  
  Same here , I haven't gotten the hang of it myself tbh it does seem like they are having issues with storage space

71 comments