Updated! Updates are shown in quote text like this. Some scores are updated following app updates.
An Apps Experiment
Cross-posted from https://lemmy.world/post/18159531
Introduction
This is an experiment I performed out of curiosity, and I have a few big disclaimers at the bottom. Basically, I’ve seen a lot of comments recently about one app or another not displaying something right. Lemmy has been around for a while now and can no longer be considered an experimental platform.
Lemmy and the apps that people use to access the platform have become an important part of people’s lives. Whether you are checking the app weekly or daily, and whether you use it to stay up on the news or to stay connected to your hobby, it’s important that it works. I hope that this helps people to see the extent of the challenge, and encourages developers to improve their apps, too.
How I did it
I wanted to investigate objectively how accurately each app displays text of posts and comments using the standard Lemmy markdown. Markdown is a standard part of the Lemmy platform, but not all apps handle it the same. It is basically what gives text useful formatting.
I used the latest release of each app, but did not include pre-releases. I only included apps that have released an update in the last 6 months, which should include most apps in active development. I was unable to test iOS-exclusive apps, so they are not included either. In all, 16 apps met the inclusion criteria.
I also added Eternity, which is in active development, although it has not had a recent update. I was able to include several iOS apps thanks to testing from @jordanlund@lemmy.world – Thanks, Jordan! This made for 20 apps that were tested.
Each app was rated in 5 categories: Text, Format, Spoilers, Links, and Images. I chose these mostly based on the wonderful Markdown Guide from @marvin@sffa.community, which was posted about a year ago in !meta@sffa.community (here).
I checked whether each app correctly displayed each category, then took the overall average. Each category was weighted equally. Text includes italic, bold, strong, strikethrough, superscript, and subscript. Format includes block quotes, lists, code (block and inline), tables, and dividers. Spoilers includes display of hidden, expandable spoilers. Links includes external links, username links, and community links. Images included embedded images, image references, and inline images.
Thanks to input from others, I also added a test to see if lemmy hyperlinks opened in-app. There was a problem with using the SFFA Community Guide that caused some apps to be essentially penalized twice because there was formatting inside formatting, so I created this TEST POST to more clearly and fairly measure each app.
In each case, I checked whether the display was correct based on the rules for Lemmy Markdown, and consistent with the author’s intent. In cases where the app recognized the tag correctly but did not display it accurately, that was treated as a fail.
Results
Out of a possible perfect 10, 6 apps displayed all markdown correctly:
Alexandrite - 10.0
Connect - 10.0
Jerboa (Official Android client) - 10.0
Photon - 10.0
Summit - 10.0
Voyager - 10.0
Quiblr - 9.5
Arctic - 9.3
Interstellar - 9.1
Lemmuy-UI - 9.0
Thunder - 8.9
Tesseract - 8.6
mlmym - 8.0
Racoon - 7.6
Boost - 7.3
Eternity - 7.0
Lemmios - 6.9
Sync - 6.9
Lemmynade - 6.1
Avelon - 5.7
Disclaimers
Disclaimers
I Love Lemmy Apps (and their devs)
Lemmy apps devs work very hard, and invest a lot in the platform. Lemmy is better because they are doing the work that they do. Like, a LOT better. Everyone who uses the platform has to access it through one app or another. Apps are the face of the entire platform. Whether an app is a FOSS passion project, underwritten by a grant, or generating income through sales or ads, no one is getting rich by making their app. It is for the benefit of the community.
This is not meant to be a rating of the quality or functionality of any app. An app may have a high rating here but be missing other features that users want, or users may love an app that has a lower rating. This is just about how well apps handle markdown.
This is pretty unscientific
You’ll see my methodology above. I’m not a scientist. There is probably a much better way to do this, and I probably have biases in terms of how I went about it. I think it’s interesting and probably has some valuable information. If you think it’s interesting, let me know. If you think of a better way, PM me and I’d be happy to share what I have so you don’t have to start from scratch.
My only goal is to help the community
I do think that accurately displaying markdown should be a standard expectation of a finished app. I hope that devs use this as an opportunity to shore up the areas that are lagging, and that they have a set of standards to aim for.
I don’t have any Apple things
Sorry. This is just Android and Web review. If someone would like to see how iOS apps are doing, please reach out and I’ll share how we can work together to include them.
Interesting to see that even Lemmy-UI does not display markdown completely correctly
For some reason, Lemmy-UI does not convert usernames to links: @gedaliyah@lemmy.world
it does, but only if you use the autocomplete feature. it’s also a bit delayed without any indicator that it’s loading.
if you type @gedal and wait a moment it’ll load @gedaliyah@lemmy.world to be selected:
Yes, I’m not sure if that is meant to be a placeholder or a substitute for native user links. What it actually does is generate markup that converts the username into a web link, which is fine for most circumstances, but not ideal. A plaintext username should automatically link to the user. This creates an inconsistent behavior between posts depending on where (and when) they were typed.
In other words, it’s a very helpful feature, but it is not recognizing and linking usernames.
What happens if you press tab or click on the suggested item at the point in your screenshot?
For me, it inserts the link at the cursor position, but doesn’t replace the bit you’ve already typed, resulting in @gedal[@gedaliyah@lemmy.world](URL).
Anyone else have this issue?
Tables are a very common markdown extension most(?) popular markdown parsers support them
for sure, but they’re neither mentioned on https://join-lemmy.org/docs/users/02-media.html nor on the linked CommonMark tutorial.
In doing this I learned that there are “correct” but also “preferred” ways to use markdown. A heading should have a space after the even though it is correct either way.
##Heading
Heading
These lines may be the same or different in different apps.
The thing of it is, if you just highlight some text and hit the heading button in the GUI, it doesn’t include the space.
I’m not sure is valid markdown (see, eg, Daring Fireball’s “original” syntax page) … and I’ve never seen it. I’ve always understood that the space was necessary, which I think makes sense for a number of reasons TBH
So …
#This does not work
This does work
I know that it works on some sites (reddit for example). Generally, it is not preferred.
Woohoo Voyager!
Oh I didn’t know it was a web app, I’ve only seen it on droidify, among other “normal” apps. It looks amazing !
I’m looking at this in eternity and seems only spoilers don’t work from the post you linked.
User and community links work properly.
I’m very excited to see a new update from eternity. It’s always been a solid app and is likely to be my daily driver again. Right now I’m using raccoon mostly, although I was pretty disappointed to see the low score on markdown
Same here, switched from Eternity to Raccoon. Maybe Raccoon’s @DieguiTux8623@feddit.it can help with better markdown support.
We were working on this but it’s a hard task and Compose Multiplatform is still not mature as a technology, there are few libraries and those which exist do not work well.
Help would be welcome, otherwise people can use Jerboa or Voyager, nobody is forcing anybody to use any app (especially if they are FOSS).
I’ve been using Mlem since week 2 and I have no idea if I’m missing anything or not. I’ve never visited any instance on anything but Mlem.
I have an iOS device and am happy to repeat your methodology! Did you have a test thread or something with all the markdowns?
On it. I found 8 apps in the App store. I’ll PM you.
Arctic, Avelon, Bean, Lemmios, Mlem, Remmel, Thunder, Voyager.
There’s a 9th, CheeseBot, but it’s $2.99 and all the others are free.
