Skip Navigation
Fediverse @lemmy.world
gedaliyah @lemmy.world

UPDATE! Now 30% of Lemmy Apps display posts accurately

Updated! Updates are shown in quote text like this. Some scores are updated following app updates.

An Apps Experiment

Cross-posted from https://lemmy.world/post/18159531

Introduction

This is an experiment I performed out of curiosity, and I have a few big disclaimers at the bottom. Basically, I've seen a lot of comments recently about one app or another not displaying something right. Lemmy has been around for a while now and can no longer be considered an experimental platform.

Lemmy and the apps that people use to access the platform have become an important part of people’s lives. Whether you are checking the app weekly or daily, and whether you use it to stay up on the news or to stay connected to your hobby, it’s important that it works. I hope that this helps people to see the extent of the challenge, and encourages developers to improve their apps, too.

How I did it

I wanted to investigate objectively how accurately each app displays text of posts and comments using the standard Lemmy markdown. Markdown is a standard part of the Lemmy platform, but not all apps handle it the same. It is basically what gives text useful formatting.

I used the latest release of each app, but did not include pre-releases. I only included apps that have released an update in the last 6 months, which should include most apps in active development. I was unable to test iOS-exclusive apps, so they are not included either. In all, 16 apps met the inclusion criteria.

I also added Eternity, which is in active development, although it has not had a recent update. I was able to include several iOS apps thanks to testing from @jordanlund@lemmy.world – Thanks, Jordan! This made for 20 apps that were tested.

Each app was rated in 5 categories: Text, Format, Spoilers, Links, and Images. I chose these mostly based on the wonderful Markdown Guide from @marvin@sffa.community, which was posted about a year ago in !meta@sffa.community (here).

I checked whether each app correctly displayed each category, then took the overall average. Each category was weighted equally. Text includes italic, bold, strong, strikethrough, superscript, and subscript. Format includes block quotes, lists, code (block and inline), tables, and dividers. Spoilers includes display of hidden, expandable spoilers. Links includes external links, username links, and community links. Images included embedded images, image references, and inline images.

Thanks to input from others, I also added a test to see if lemmy hyperlinks opened in-app. There was a problem with using the SFFA Community Guide that caused some apps to be essentially penalized twice because there was formatting inside formatting, so I created this TEST POST to more clearly and fairly measure each app.

In each case, I checked whether the display was correct based on the rules for Lemmy Markdown, and consistent with the author’s intent. In cases where the app recognized the tag correctly but did not display it accurately, that was treated as a fail.

Results

Out of a possible perfect 10, 6 apps displayed all markdown correctly:

Alexandrite - 10.0

Connect - 10.0

Jerboa (Official Android client) - 10.0

Photon - 10.0

Summit - 10.0

Voyager - 10.0

Quiblr - 9.5

Arctic - 9.3

Interstellar - 9.1

Lemmuy-UI - 9.0

Thunder - 8.9

Tesseract - 8.6

mlmym - 8.0

Racoon - 7.6

Boost - 7.3

Eternity - 7.0

Lemmios - 6.9

Sync - 6.9

Lemmynade - 6.1

Avelon - 5.7

More details of testing here

213 comments

  • Hey, I'm the Photon dev. I'd like to know which parts Photon incorrectly displayed, so far I only see tables rendering incorrectly. I'll have this fixed soon.

    Update: fixed table displays, pushed to main

    Could this be updated now? 🥺 (you can test here)

    • Holy shit, Photon has gotten this good now? When I tried it a few months back it felt like just yet another Lemmy client. Now it feels so smooth and polished. Works great on mobile even. Thanks for making this!

    • Unrelated, but photon keeps randomly redirecting pages to what is previously viewed. It has screwed me over by making me post to the wrong community.

      • I don't understand what this means. There are no redirect calls at all in photon other than for /comment urls, and certain layouts.

    • Photon is so great i honestly feel like it should replace the default

      • Agreed, translating it to french made me discover so many little features, did you knoe it can show the political bias of a linked article?

    • Hey, the admin of slrpnk.net has been thinking of making Photon the default frontend but updates to it sometimes cause breaking issues? Any chance you could get into contact with them so it can become the default in a way that updates wont break it?

  • Is there a list of what each app failed? It would be nice for the devs to be able to see. I use Mlem, and there is about to be a new release rebuilding it from the ground up. Hopefully it will rate higher once that happens.

    • Yes, I've linked it in the post, and you can find the test post and detailed results.

      • Thanks. Interesting how the apps, even those that have lower scores, perform better than a web browser. Using Safari and Firefox (on a laptop) and both open your links in Lemmy.world instead of that thread on my instance. Neither recognize the user as anything other than text.

  • This is awesome, thank you. I switched from memmy (iOS only) to voyager because it doesn’t display code blocks properly (usually doesn’t even show what’s in them) so reading certain posts or comments about computers or programming was a disaster.

    • Memmy was the first app I used, but it is abandoned now, sadly. But Mlem is actively being developed. I have not tried Voyager yet.

  • If you're getting that granular then you must've had to record the data somewhere. Did I miss where the OP is sharing their data set?

  • Btw I just found out that lemmy-ui supports markdown citations.

    It’s not documented AFAIK though.

  • One note on Jerboa, at least for me gifs don't seem to play when embedded in comments. Otherwise 10/10 for me.

  • @gedaliyah@lemmy.world

    I have an iOS device and am happy to repeat your methodology! Did you have a test thread or something with all the markdowns?

    • Yes, I used the Markdown Format Guide linked above. For user and community links I used this comment, and for inline images I checked the FOSS icon here.

      If you PM me screen grabs, I'll add it.

      • On it. I found 8 apps in the App store. I'll PM you.

        Arctic, Avelon, Bean, Lemmios, Mlem, Remmel, Thunder, Voyager.

        There's a 9th, CheeseBot, but it's $2.99 and all the others are free.

  • Left this comment in the other thread too, but posting here for visibility:

    Quiblr should now have each of the markdown criteria fixed. Huge thanks for the feedback and for all this analysis. Consistent markdown is important for a great and consistent user experience across the lemmy ecosystem

    • Great! Thank you for the gorgeous app. I really love the style, and I think the personalized feed is brilliant.

      On my device, the lemmy hyperlink in the test post is still opening outside the app. I'm not sure how other web apps handle this but it would be the only additional change that would make it a perfect score.

      As an aside, I would love to see it as a PWA or standalone app. I don't know if that's on your roadmap but I think it would be neat.

      • Thanks for the Quiblr compliments! And Post and Comment links should now open in-app. I think that covers everything

        And Quiblr is a PWA. Native apps are in the works

  • test ~test~

    The above feels wrong but idk if Lemmy has a formal markdown spec. I haven't had time to dig into it. This is what it looks like in Jerboa. If it wasn't 6 AM I'd try to file a big report.

  • There are a lot of image/gif(?) posts that I haven’t been able to view either on the Memmy (Apple) app or in-browser with either Safari (Apple) or Google Chrome. I imagine it comes down to the file types as well as the lack of native hosting to standardize posts of different media types, but I’m not the techiest person to consult on that. One downside of the fediverse is the lack of standards for file hosting/conversion/displaying to ensure that all media can be accessed regardless of the browser/app (or, alternatively, the lack of an all-encompassing app for all devices [Jerboa sounds like the closest to this to me but it is not available for iOS yet]), as well as the self-funded nature of the instances commonly not having the budget to natively host multimedia content such as videos.

  • So what's the technological story here? I'm guessing lemmy itself uses a particular markdown parser that could probably be extracted and used in other contexts as it's likely written in rust and should therefore be pretty portable without too much effort.

    Are other apps just using whatever markdown parser is convenient to them? Is this something that the lemmy and threadiverse community could converge on? Even the fediverse as a whole where just about every platform other than mastodon supports writing in some for of markdown ... feels like a pandoc like utility could go far.

    • I'm probably not the person to ask, to be honest. Lemmy as I understand it is the protocol that exchanges the information about posts, etc. The post content is stored and shared as plaintext, but Lemmy also has instructions about how a UI should interpret the text and serve it to the user.

      Ideally, the same text should appear consistent across any UI. Obviously, some apps will use different fonts and colors and may interpret the style of an element differently.

      • Ideally, the same text should appear consistent across any UI. Obviously, some apps will use different fonts and colors and may interpret the style of an element differently.

        Oh yea styling isn't the issue here ... it's whether the markdown is correctly interpreted and rendered. AFAIU, lemmy doesn't have any instructions about how to interpret the text, just some standard that they've chosen to use, along with their open source software for doing so (as they've built too clients, the default web UI and Jerboa).

213 comments