Your explanation of federation that it "works like email" will be misunderstood
The linked post shows how most non-tech people's understanding of email is very very different from most of the people here.
You're viewing a single thread.
In my experience, the majority of people doesn't have the slightest clue how mail works. Somehow you type it in and provide it with an address into one of the three indistinguishable fields that are titled "To", " CC", "BCC". And by some black magic it either appears on the screen of the other person. Or it doesn't. That's about the amount of knowledge.
So comparing something to this is kind of meaningless.
Yup, and people younger than a certain age think email is as archaic as the pony express.
Hilariously, fax machines are as archaic as the pony express. They were invented around 1850.
Abraham Lincoln could have literally sent a japanese samurai a fax.
And we still don't have any worthy successor. In contrast to the pony express.
As a younger tech person, I definitely don't get a lot about email. It's old and weird and arcane and half it's features that match newer services seem to be built on top of hacks that are enforced through convention alone that will break if I decide I like to format my titles a little differently. Third party clients work, but the main providers, Gmail outlook use some proprietary api to make sure their own works well while everyone else gets stuck with shitty imap. There's endless little incompatibilities. It all just feels like delerict tower held together with miles of duct tape. Oh and I still haven't found a good answer to why calendars are so tied up with email.
In my country, people above a certain age think sending an email is impressive
Ironically it is often that way with Lemmy as well, outside of Lemmy.World. You write up a post, maybe it makes it to its destination, maybe it does not (for several days, after which point extremely few people will see it when it suddenly appears, but down among the older content no longer listed as "new"), maybe people write comments into it, maybe they don't but who even knows if you aren't able to see any of them to be able to respond.
This is definitely a monthly or almost weekly occurrence, even if not quite a daily one, though it depends strongly on what instance you are on.
Also, whether it makes it to the destination or not depends on which server you try to view it from.
And I haven't even begun to start into the defederation artifacts yet!?
Right. Learning Lemmy and the Fediverse takes some effort. The onboarding isn't super smooth and flawless... I think we all know this. I'd still like to see a few design changes. I think generally we're headed in the right direction. Albeit kind of slowly.
I haven't noticed any federation hiccups in quite some time. There was some debacle with two updates. But since then it's been forwarding posts within seconds for me. At least on the last two instances I used.
Almost every single one of my posts has had major issues. Even those from other instances. e.g. Rimu messaged me about this one that did not federate for 2-3 days, and consequently was seen by very few people. And here's one from a difference instance that made it to its destination on !tenforward@lemmy.world, but from its originating server I could see none of the comments, and had to respond from a third account involved in that 3-way attempt at communication. (a post talking about such federation issues on that same server) So to be very clear, I am not saying that instances running PieFed software are having issues, but more that the issues are with Lemmy regardless of software type run.
Uh, right. Sorry, I did not notice you were from Piefed... I was talking about the times when we had the borked Lemmy updates... Did you ever debug or resolve your issues? Is there a way to tell something didn't federate? And is this an issue specific to Piefed? Or to the whole Fediverse? I'm not sure if I'm affected. I occasionally check my posts from another account and it always seems okay. But I mean I don't do it very often.
Yes often I've chased it down to some small degree. The one that Rimu messaged me was near a time of great instability on piefed.social and I got a bunch of gateway errors even so much as trying to reach it as a user (having nothing to do with posting I mean). This instance has calmed down a ton since then and works perfectly these days, but things happen from time to time. Likewise the incident with StarTrek.Website that I mentioned and provided a link describing more.
Other occurrences have still other causes - e.g. a few of them in one community seems to have been due to my posts getting "locked". I have no idea why - perhaps the new mod was just fat-fingering the button? I did not ask. But now the vote counts vary GREATLY (192 vs. 183 vs. 98 vs. 0 etc.) depending on which instance you view it from. If you want to test for yourself, a good one to use is https://piefed.social/post/330559 - though I notice that (fairly recently created) community !tech_memes@lemmy.world does not appear at all on your instance.
The primary cause though was a limitation in how the ActivityPub protocol was implemented in the Lemmy codebase, and not having anticipated that ~80% of the entire Lemmy-based Fediverse would concentrate itself onto a single server, Lemmy.World. So how it works is that any "action" - a post, a comment, an upvote or downvote - will be federated out to all the other instances world-wide at a rate of 1 per second. However, if the ping from the other servers to Lemmy.World is itself a significant approximation of that, then the list of actions to be federated will fall behind and take longer to catch up. Eventually after more than a week it gives up entirely, but in the meantime an action can be delayed for days. Poor Aussie.Zone - geographically distant from the EU - has been really having a hard time of it ( https://aussie.zone/post/13429731 ).
Fortunately this problem has already been fixed in the Lemmy codebase by allowing multiple actions to be sent in parallel (https://github.com/LemmyNet/lemmy/pull/4623) - however, what causes the continued problems nowadays is the fact that Lemmy.World is still awaiting that upgrade to 0.19.6 to make use of that change in the codebase (release notes) (actually now 0.19.7 is already out too, having come less than a week after the former, and representing just a few bugfixes, release notes). When Lemmy.World upgrades to one of those, a good deal of these systemic issues should calm down, by a GREAT deal, if not entirely.
Afaik, there is nothing particularly special about instances running PieFed having troubles connecting to any Lemmy instances. In fact it seems rather stable compared to many (even most!) others - particularly StarTrek.Website that has poor uptime. In fact, https://piefed.fediverse.observer/list reports that piefed.social has a remarkable uptime rate of 99.89, which I very much believe, compared to the aforementioned StarTrek.Website's rate of 98.20, although a year ago when I left it it must have been significantly poorer b/c it would be down for days sometimes, and every single action took like a minute sometimes, back then. Your own instance reports 98.60 - does that sound right?
Rather, it is Lemmy instances - particularly smaller ones (e.g. https://lemmings.world/post/14171987) - having trouble federating specifically with Lemmy.World.
And then recently there were a bunch of instances having troubles connecting to lemmy.ml too (https://lemmy.world/post/22196027) - though this one is more expected as that one is administered by the developers of the Lemmy codebase, and thus that is the place where they test out all of their new code in beta, prior to deploying it across the entire Fediverse. Sometimes that leads to some REALLY odd behaviors, such as entries disappearing from modlog files that were extremely concerning to people, but it is par for the course with that highly special instance, which is unique in its manner.
Edit: ah and I neglected to answer one of your questions: as you said, the way to tell if something federated properly or not is to check the instance - specifically the one hosting the community that you are sending it to. So e.g. to check a post to !tenforward@lemmy.world, I would visit Lemmy.World. If it is there but not on your home instance, then at least that particular message packet got sent, even if the message packet from Lemmy.World to your instance got lost or fell behind in its processing backlog somehow.
Lemmy.World is still awaiting that upgrade to 0.19.6 to make use of that change in the codebase (release notes) (actually now 0.19.7 is already out too, having come less than a week after the former, and representing just a few bugfixes, release notes).
On the other hand, 0.19.7 still has a picture issue: https://github.com/LemmyNet/lemmy/issues/5196
Ah, so is this what happened to lemmy.cafe?
And will Lemmy.World wait until that is fixed do you know? MrKaplan said perhaps a week ago that they were not ready to announce any plans, but perhaps you've heard something since then. I don't know how stable 0.19.6 itself is in that case, or if they need to first wait until 0.19.8 and then wait further for the bugs in it, like 0.19.7 (and 0.19.6 too), to be discovered and patched.
Edit: in any case, thanks for sharing!:-)
Thank you for all the detailed explanation. Seems I wasn't aware of lots of current events. Especially regarding Lemmy. I always thought it can't be too hard syncing posts to 45k (monthly active) users... I guess there's still some way to go.
By the way, Piefed does not pull in remote communities by default. It only does it once a local user subscribes. And that's why a lot of them are missing on my own instance. I skipped quite some of the meme communities when I switched. And I'm already trying to foster smaller instances. I don't subscribe to communities on lemmy.ml and I'd like to have an alternative to lemmy.world. But you're right. A lot of the activity here happens on lemmy.world and we can't do without.
And last Lemmy instance I used was discuss.tchncs.de which always seemed fine. But it has a very capable admin and is located in Germany, so probably not too far out.
My uptime should be less than other servers. I'm using it for testing and development. And sometimes I break stuff and it's down until i figure things out.
By the way, Piefed does not pull in remote communities by default. It only does it once a local user subscribes. And that's why a lot of them are missing on my own instance.
Oh yes absolutely - and Lemmy is the same way, so this too is not something specific to PieFed. For instance, that post I mentioned previously where my vote counts are all over the place, when viewed from StarTrek.Website has zero comment associated with it, and no upvotes beyond the default - and I have another post that is the same way, though it seems like later that same day someone subscribed and from that point onwards the community starts to have comments and some of the votes seen from other instances.
Or perhaps it is that these posts were locked somehow / for some reason? Which looks to be accidental from a new, inexperienced moderator in this brand-new community, and it was reversed a couple of days later - although that fact again depends on where you look. With an account at StarTrek.Website, I look at the post moderation history and it says that it is still locked: https://startrek.website/modlog?postId=16510256. However, with an account on DiscussOnline (substitute with whatever other alt you may have - hendrik@lemmy.ml?) I see that not only was it unlocked, but that unlock event happened 7 days ago: https://discuss.online/modlog?postId=13575162.
Even so, the post when viewed from DiscussOnline shows 98 upvotes and 8 comments, but when viewed from Lemmy.World (where the !tech_memes@lemmy.world community is located) it shows 191 upvotes and 9 comments (or I think it's 193 upvotes and 2 downvotes, but the web UI no longer shows those individually, unless you jump through many many, undocumented, hoops - e.g. I think I can see those broken down into their individual components on a mobile device in Firefox, possibly solely when viewing a individual users list of posts but not when looking at a post directly or in the standard community view, and definitely you cannot see this breakdown from either Chrome or Firefox on a desktop, etc.). And since it has been 7+ days, this is now enshrined in stone, and we can be confident that having not caught up by now, it never will. A decade from now, if e.g. DiscussOnline is still with us, it will show this post as having 98 upvotes rather than the true value of 193, and StarTrekOnline will still show the default upvotes=1 and no comments, thus providing 3 different stories for this same identical post, depending on how you try to view it - and only one of those stories being explanable by the fact that nobody on StarTrek.Website had subscribed to the community yet (MAYBE, b/c there are 2 other posts that are even older in that community, which have +1 upvote added!? so perhaps this is a complex mixture of that + the locking effect, with the unlock action having not been propagated correctly).
The above stories reveal - federation is NOTHING AT ALL LIKE EMAIL. In the latter, the message either gets passed or it does not, whereas in federation, you can see partial messages as I've shown. And this has not even begun to delve into the variety of defederations that further complicate any mess - especially with a unidirectional defederation where one account can talk to someone on a server that has defederated from them, though the recipient will never be alerted to that fact nor have the capability to respond. Thus it is my opinion that trying to fit the square peg into the circular hole is never going to work - the email analogy is hopelessly simplified, so much so that as soon as users begin to encounter such complexities when they make their posts, especially the content creator types that we very much want to come here, they may outright leave, and moreover be very vocal about how we are not what was promised to them. So while we could say "it's a little bit like sending email", I don't think we should push too hard on that avenue, making it sound so simple, b/c it's really not!