1y ago

Lemmy instances that are focused on mirroring Reddit content?

I've posted before about my fediverser project, and I am now looking to see who is interested in participating.

The short description is that it does the following:

it runs a lemmy instance which will be the home of bots that mirror accounts on reddit.
The admin of this instance can choose what subreddits are going to be monitored from this instance. Let's say that these are the "source" communities.
For these selected subreddits, the admin can define where the posts from these subreddits should be posted in the other lemmy instances. We can, e.g, map posts from /r/selfhosted to !main@selfhosted.forum or !selfhosted@lemmy.world .
You can choose whether to mirror the posts only or the whole thread with comments from reddit. Each of these will be authored by the account that mirrors the original reddit user.
(WIP, optional) responses to the reddit mirror accounts will create a comment on reddit with a link to original lemmy thread.

So, now I finally got to deploy the first lemmy fediversed instance, and I'd like to know the following:

which subreddits you still follow but would like to bring to the fediverse?
For instance admins and community mods, what communities you would like to be the destination of the mirror posts, and would you be interested in having the posts only or the whole thread?

Bear in mind that this is NOT advised to be done for the bigger subs. The idea here is not to create a huge army of bots and overwhelm the fediverse, but mostly to create a migration path to those who rely on the more niche subreddits.

128 comments

Nobody wants to participate in a disconnected discussion. We have multiple instances with bots that do precisely what you're saying, and they just flood the feed with posts that people won't engage with.
Nobody wants to have a discussion about a reddit post. Nobody wants to have a discussion about a hackernews or slashdot post. If Lemmy just looks like a place to mirror Reddit content, people will see that and just go to Reddit and engage directly.
We need more people posting to Lemmy. If this is just a place for bots to have discussions with themselves, nobody will stay here very long.
- Precisely, a lot of us left that horrible place to escape all the rampant transphobia and persecution they have over there. We don't need their kind of "content" mirrored here. My first thing I'm going to do when they allow us to block instances is to immediately block those that have mostly repost bots.
  I think Lemmy as a whole should forbid any links from Reddit just protect everyone here. Is it censorship? Yes, but it's for a good cause.
- Nobody wants to participate in a disconnected discussion.
  This is why the next part of the work is to build a bridge to send notifications to the people on reddit.
  We have multiple instances with bots that do precisely what you’re saying
  Do we really? Can you point me to them?
  Nobody wants to have a discussion about a reddit post.
  You are starting to sound like a gatekeeper. First, this is not the goal of this tool. The idea is eventually to have mirror accounts ends to work as proxys to allow for two-way bridges. Second, a lot of people know that they are not with reddit, but when they have come to lemmy they bounced back because they couldn't find the content they were used to.
  We need more people posting to Lemmy.
  Agree, but If we follow the rule of 90/9/1 for lurkers/commenters/posters, this means that we can bring 90% of reddit's userbase to the fediverse just by bringing the content here. The posters and commenters will eventually follow.
  The logic is simple: unless I start breaking into people's phones and computers, I can not force people them to post to Lemmy, but I can get their content here. A tool like this can help the intolerant minority to drive the behavior of the majority.
  
  If we're interacting with Reddit anyway what's the point of using Lemmy
  
  Okay not precisely, but we have a bot (I think it's the one at smeargle.fans) that reposts Reddit threads and replicates all of their comments, which nobody engages with
  You are starting to sound like a gatekeeper.
  Well that term just doesn't apply. I'm not saying "Real Lemmy users avoid anything to do with Reddit" or anything along those lines. You asked for feedback, and I gave you my honest criticism of it.
  I understand that you found a project that sounds fun to make, and it probably will be. This is what we engineers do, we get excited to build things that seem to have clever technical answers. However in my past few months on Lemmy, I have seen these ideas, and have seen the way they tend to work out so far.
  The logic may be simple, but human psychology is rarely as simple as engineers wish it could be.
  Feel free to build your project. All aspiring engineers should make things that they want to make. But if you ask for feedback, don't argue that the feedback is wrong. Not all solutions end up working out the way you hope, and that's part of the engineering life. And based on prior experience, this one is likely to get the same treatment that the other repost bots get.
  
  Do we really? Can you point me to them?
  lemmit.online did this and was recently defederated, partially because the one admin wasn't able to prevent the volume of posts being generated from ending up filled with spam.
  There's a few HN bots too, but unfortunately the posts never get much engagement here, so I have to go through to HN to read some discussion about it (which I will do, but a much bigger proportion of people just don't appear to engage with mirrored content here)
  
  You lost me at calling the poster a gatekeeper.
  
  We have multiple instances with bots that do precisely what you’re saying
  Do we really? Can you point me to them?
  Two examples:
  https://lemmit.online/u/bot
  http://zerobytes.monster/u/bOt
  I wonder if the time people had to spend for removing their spam from their feed already surpasses the time the developers have spent setting them up. I know, I know, you intend your bot to be different, so not 'precisely' that, but I'm worried the result might be the same. Lots of posts with zero engagement, which give people the impression Lemmy consists of bot posts, ultimately driving user engagement down.

Honestly the fist thing i did after join Lemmy was to block any bot in the setting and filter off all "Reddit something" groups. Just my preference ofk.
- Fair enough. One of the design reasons that I am working from a separate instance for the bots is that it makes it easy for admins to block in case they are worried about being flooded.
  
  Honestly, I like this idea, just because it means I could block your instance in my app and instantly filter out that kind of content, just like how someone can block lemmynsfw to get rid of almost all porn.

These bots merely advertise for reddit. They drown the All feed in zero engagement posts, hiding actual activity.
Experienced users might be able to handle this (although the question persists why one unresponsible bot admin should be able to force thousands of users to take action), but new Lemmy users will not look for a fix. They will leave and never come back.
These bots make Lemmy a worse and shallow copy of reddit. Please stop running them, or make it in instances which do not federate their content to the rest of Lemmy.
- They drown the All feed in zero engagement posts, hiding actual activity.
  If I tell you that I've been running this system for two days already, have you noticed and/or found yourself "drowned" by anything?
  
  Yes, every time I try to use the All feed, I'm presented with new communities which consist of nothing but zero-engagement bot-posts.
  Every time, I can choose between wasting more time on blocking them or abandoning that Feed altogether.
  Granted, not your bots as far as I can tell, but two days is not a long time either. Give it more time and into more hands, abuse will happen.

There's an entire instance for that already - lemmit.online and it sucks. There's no engagement, no interaction. Nobody wants it. Everyone hates it.

Why isn't the focus aimed at creating more communities over here, that actually add to engagement & interaction...if I want to see what's happening over there, I'll go and take a look... I'm one of those that has bots blocked anyway, so makes little difference to me really..
- Why isn’t the focus aimed at creating more communities over here, that actually add to engagement & interaction…i
  Because it's not mutually exclusive and we can do both.
  if I want to see what’s happening over there, I’ll go and take a look…
  Yeah, but what about those who do not want to go over there out of principle. I know I am not the only one in this case, and I'd also like to have a way those who want to migrate.

If you are doing this PLEASE do it on your own instance, so we can just block the instance if we want to and you don't waste someone else's bandwidth
- Yes, all bots will be running on the same instance, and I will only send mirrored posts to communities if I get explicit approval from the mods.
  
  thank you

I'm not a fan of mirroring Reddit, but I can't stop you. Make sure it is easy to block (such as one bot user posting all posts but not comments) if you don't want to be defedded from a bunch of instances.
- Actually, the idea is to have all bots users on the same instance precisely to make it easy for unwelcoming admins to block or defederate it.
  
  Edit: ignore my post below. I see the point of your project now (help bootstrap niche communities.) I just hope admins use it as intended.
  You're giving choice to admins but not to instance users with this.
  Blocking nsfw communities from nsfw instances was already annoying and tedious (and I'm far from being a prude, I just don't want to see "cum in my creampied pussy" when I'm browsing while having lunch.)
  Reddit content is mostly rage bait these days. This is why many of us fled that cesspool.
  Why not use some other website instead of reddit?

I think it took me less than a minute to block the lemnit bot.
We want to grow beyond just being a Reddit clone/replacement - mirroring active discussions here just feels like stalking an ex on Facebook.
That said, in a previous discussion about about archiving good answers from Reddit, I did suggest that this would be a great use for a wiki that was integrated into Lemmy. Being about to semi-automate the retrieval and formatting would be useful. I think starting new threads for them isn't the way to go.
- mirroring active discussions here just feels like stalking an ex on Facebook.
  I am not sure if that is the best analogy. We didn't break up with the communities in the subreddits, we broke up with Reddit, Inc.

Interesting initiative!
FYI, there is already https://lemmit.online/
- Yeah, I know of lemmit but AFAIK it has the opposite ideas in many cases:
  It seems to be archiving the most popular instances, not so much the niche ones
  It is going to mirror posts only, not the discussion.
  All posts are created by the same bot account, so we lose information about the original content.
  I wonder if the dev from lemmit would be interested in being the host of some of the "fediversed" communities, so that we can have comments as well.
  
  You have it follow whatever instance you want it to follow. There’s a request community that you post to and it’s then added to the queue.
  I don’t understand the point of this at all. Might be a fun project for you to do, but nobody wants a bunch of communication set before them that you can’t interact with. It doesn’t help this grow at all.
  
  Something a bit similar to what lemmit is already doing, but more powerful with your addition of comments: read-only, best-of archives of really old content from popular subs.
  10-5 year old askreddit posts for instance would be interesting blasts from the past to read today. Isn’t there already a ‘best of Reddit’ convention on Reddit itself that resurfaces such content from time to time?
  
  Wait, you're intending to use multiple bots, so users have to block multiple spammers?

Love the idea for subreddits I can't find a Lemmy sub for. But I don't want all the rest. I specifically went out of my way to block seeing bots on Lemmy. I don't think there is a way to opt in to seeing just one and even if there was I don't see how I would transition that into only seeing bots for the subs on reddit I'd still want to engage with. I don't understand how you're planning to manage this?
- I am not sure what you mean by "this" that I am supposed to manage. If you don't want to follow bots, you won't see the mirrored account.
  
  Right. But what I am saying is that even if I turned the setting off and was seeing the bots how would I only see the bits and pieces I wanted from reddit and not the whole of the reddit website mirrored?

I see what you’re trying to accomplish and congratulate you for trying to make Lemmy a better place, but I don’t think it’s a good idea. As many people here I’ve blocked @bot@lemmit.online precisely because it just posts Reddit content and floods my feed, with what is basically spam. Also many people came here just to avoid Reddit and to try and make something better. Sure, when 0.19.0 comes out and people will be able to block instances, lemmings will have a choice if they want to see that instance, but what about new users, or people just checking Lemmy out? Do we really want them to see reposted reddit content? Because they can already do this on reddit. What we need is to stop being so invested in “our ex” and just grow as a community, in a natural way.
- The argument can be made that bots measuring the content are no better than some random dude on the Internet reposting shit they like. Situation becomes worse when that same "bot" doesn't credit the author proper.
  
  Memes are meant to be shared, and it’s a great way to grow small communities. Karma removed is not that big here since reputation points are not shown (at least in the webUI). So a person will post a few memes and could start a conversation, something a bot can’t do.
- it just posts Reddit content and floods my feed
  Wait, how come does it flood "your" feed if the lemmit bot only posts to their own instance/communities? If you are browsing with the "all" view, it's not really your feed.
  Apologies for the "you're holding it wrong" response, but maybe it would be better if you just start browsing the specific communities that you want to follow?
  
  It’s seems like you ignored my point entirely just to argue on a technicality. As I asked, should Reddit content reposted by bots be something a new user should see?
  Also browsing /all is a great way to discover new communities, but even if you don’t browse it bot communities find their way into the discover communities tab, which makes it more difficult for people find the communities that they want to follow.

What if I don't want the content I post to Reddit mirrored on Lemmy? How can individual users opt out?
- The plan is to let users authenticate to the mirror instance with the mirrored account, which would cause the bot to be disabled. But that is not implemented yet.

Let's say I have a favorite sport and there exists a sub_ named: r/.
Let's also say there already exits a Lemmy community and that community is struggling to get off the ground: !@lemmy.world
I can see a value add if your project directly helps !@lemmy.world get started; but I don't see how it does. If anything wouldn't your project compete with !@lemmy.world and therefore hinder it?
It might be different if your project directly tied r/ to !@lemmy.world but it doesn't.
- So, like /r/soccer and !main@soccer.forum?
  
  Yes
  
  Checking out !main@soccer.forum I saw very few posts by bots. Mainly saw posts by you. I saw one post coming from alien.top .
  What's interesting is that only posts by bots have any comments. So maybe this could be a good way to get communities started.
  Therefore, if it's okay with the admins at the following community, I'd nominate !tennis@lemmy.world
  There's almost nothing happening there.
- Yeah, check !main@soccer.forum, I am using the tool to post some content that I find on /r/soccer. I am working to do similar things with /r/NBA and /r/NFL.
  
  Did you check with the admins on lemmy first or are you bot posting without permission?

Are you going to foot the bill for the Reddit API fees?
Have you considered what you will do if Reddit cuts you off? IANAL, but it's fairly clear from the TOS that they will likely shut you down.
Except and solely to the extent such a restriction is impermissible under applicable law, you may not, without our written agreement:
license, sell, transfer, assign, distribute, host, or otherwise commercially exploit the Services or Content;
modify, prepare derivative works of, disassemble, decompile, or reverse engineer any part of the Services or Content; or
access the Services or Content in order to build a similar or competitive website, product, or service, except as permitted under any Additional Terms (as defined below).
https://www.redditinc.com/policies/user-agreement-september-25-2023
- I am not mirroring all of reddit, and the idea is that each "fediverser" instance pulls only enough data to stay under the 600 requests/ 10 minutes, which is free.
  If they decide to go after these fediverser instances, they will have to play whack-a-mole, because anyone can get new keys and start anew.
  
  It'll be easy enough for them to block fediverser/0.1.0 user-agents, so perhaps that's not a safe default value since it's an easy target.
  I'll be honest, I saw a previous post of yours and was sceptical, but I think based on the idea, you've taken the best steps to make this a reality. Having the communities be part of instances where they fit in and can be maintained by moderators who care about the subjects is a challenge, but it does set it up for longer success.
  As others have pointed out, there's still an imbalance where people don't realize they're replying to shadow accounts (like this for example). Maybe a good solution would be to DM someone who replies to a comment by one of the bot accounts explaining what's going on. Maybe asking the person who commented to reach out to the user on Reddit directly, and asking them to join the Fediverse would be a good solution and would bring in the human element to the process. This would avoid you having to build that feature (and likely appear to be a spammer) which might have a higher conversion ratio.
  I'm not sure if you have a plan for it, but somehow allowing the Reddit user to take over the shadow account would probably achieve your goal of getting more people to convert, and would be a benefit to niche instances looking to grow their organic members. However you do this, it should be seamless to the new member, with the minimal number of hurdles.
  Good luck!

Isn't this like stealing, guys? No, not from Reddit - from authors. Ask for a consent before mirroring anything, for the love of Fediverse. Cheers!
- The better question would be whether this constitutes fair-use. Because it's certainly not stealing.
  
  TINLA: factors for fair use don't seem to align, though.
  Such use does not characterize commentary, parody, etc. and is not transformative.
  Post may prove to be substantial on its own, especially if it's an art piece.
  Most of the work (individual post) or crucial parts being used.
  Since there is most likely no thorough link to the author's website or profile, they lose the audience - nobody will go to look up the same post twice, not through Google and Google Images, especially.
  About that last point: solvable by manually gathering authors' links or making a hyperlink to respective Reddit profiles.

What an interesting idea. As this project scales, how would you think of getting around the Reddit API limit problem? This sounds pretty API intensive. I also wonder if Reddit might see this as a TOS violation (particularly when the bot was posting comments) and killing it without even reaching an API limit.
That said, I applauded you for trying to think of creative ways to increase content on Lemmy. One thing in particular that I miss are the questions on niche subreddits, particularly hobby subreddits. You can learn so much just by reading others’ questions. Lemmy doesn’t have the user base and reach to support stuff like that yet, so I like that you’re trying to think of ways to increase that content here.
- As this project scales, how would you think of getting around the Reddit API limit problem?
  The first idea is to scale horizontally. More instances run by different people, each of them running for different communities.
  The second idea is a bit crazy but I would only be able to do it with some serious financial support and the help of a mobile app developer. Basically it would require a mobile app that could work as a client of both reddit and lemmy.
  I also wonder if Reddit might see this as a TOS violation.
  I do worry about it, but if it gets to this point, it would mean that this project would have started to make some noise. If it has started to get their attention, it would mean that the fediverse would be already reaching some critical mass.
  I miss are the questions on niche subreddits, particularly hobby subreddits.
  That's exactly the type of community that I want to bring via alien.top. I started !makers@communick.news, but it didn't catch on. Can you give me a list of specific subreddits you want?

This is silly.

This is a great use case for a lot of public health related subreddits and for niche advise subreddits like bdsmadvice, askdoctors, masks4all, etc.
I really hope the bridge feature is implemented soon!
- niche advise subreddits like bdsmadvice, askdoctors, masks4all, etc.
  One of those things are not like the others.... XD
- Yeah, thank you. Any type of support helps.

@feddit.de if possible :) There is also @pathofexile-discuss.com for poe

Oh and please also add /r/manga
- That's the only thing bringing me back to reddit every once in a while. As someone who pretty much just reads from an app /r/manga was amazing for finding new series.

I like it, seems like a good way to get to critical mass on the small subs here. Can't really offer any help, but good luck and thanks!

can you post links to all communities you create when you finish please :)
- I will update this list as it goes:
  Currently created:
  !fcbayern@soccer.forum
  
  r/okbuddyretard would be nice

Subreddits id like to have mirrored with discussions so i dont have to visit reddit anymore.
/gakinotsukai /90dayfiance /pathofexile /linux /fcbayern /diablo /diablo2 /diablo2resurrected /diablo4 /hextcg
- Great, thank you so much for the suggestions! Question now is, where should I point them to?
  I can have /r/linux at communick.news, and the football ones I can gladly send to soccer.forum. What about the gaming ones?
  
  @feddit.de if possible :) There is also @pathofexile-discuss.com for poe

Thank you for this. Don’t listen to the haters, by bringing reddit’s content over here you’re still adding more content to the fediverse. I would love it if you brought over meme communities
- Thank you for the kind words, but I'm probably not going to do this for meme communities for the following reasons:
  lots of images means hosting a lot of content and a big storage bill.
  lots of users with lots of comments/submissions, will be hard to mirror those without hitting API rate-limiting ceilings.
  if the mirrored instance gets lots of bot accounts who post nothing but low-quality content, it will make the people here on Lemmy associate it with spam.
  But if you really want those, you can get a domain name and I could perhaps host it for you on https://communick.com ?
  
  This is my first time looking into this service, but I would be willing to pay to have the service hosted. Does your link above include a way to get started with running the software you coded?

128 comments