Soliciting Feedback for Improvements to the Media Bias Fact Checker Bot

My personal view is to remove the bot. I don't think we should be promoting one organisations particular views as an authority. My suggestion would be to replace it with a pinned post linking to useful resources for critical thinking and analysing news. Teaching to fish vs giving a fish kind of thing.

If we are determined to have a bot like this as a community then I would strongly suggest at the very least removing the bias rating. The factuality is based on an objective measure of failed fact checks which you can click through to see. Although this still has problems, sometimes corrections or retractions by the publisher are taken note of and sometimes not, leaving the reader with potentially a false impression of the reliability of the source.

For the bias rating, however, it is completely subjective and sometimes the claimed reasons for the rating actually contradict themselves or other 3rd party analysis. I made a thread on this in the support community but TLDR, see if you can tell the specific reason for the BBC's bias rating of left-centre. I personally can't. Is it because they posted a negative sounding headline about Trump once or is it biased story selection? What does biased story selection mean and how is it measured? This is troubling because in my view it casts doubt on the reliability of the whole system.

I can't see how this can help advance the goal (and it is a good goal) of being aware of source bias when in effect, we are simply adding another bias to contend with. I suspect it's actually an intractable problem which is why I suggest linking to educational resources instead. In my home country critical analysis of news is a required course but it's probably not the case everywhere and honestly I could probably use a refresher myself if some good sources exist for that.

Thanks for those involved in the bot though for their work and for being open to feedback. I think the goal is a good one, I just don't think this solution really helps but I'm sure others have different views.

One problem I’ve noticed is that the bot doesn’t differentiate between news articles and opinion pieces. One of the most egregious examples is the NYT. Opinion pieces aren’t held to the same journalistic standards as news articles and shouldn’t be judged for bias and accuracy in the same way as news content.

I believe most major news organizations include the word “Opinion” in titles and URLs, so perhaps that could be something keyed off of to have the bot label these appropriately. I don’t expect you to judge the bias and accuracy of each opinion writer, but simply labeling them as “Opinion pieces are not required to meet accepted journalistic standards and bias is expected.” would go a long way.

You don't need every post to have a comment basically saying "this source is ok". Just post that the source is unreliable on posts with unreliable sources. The definition of what is left or right is so subjective these days, that it's pretty useless. Just don't bother.

I think it should be removed

My personal view is that the bot provides a net negative, and should be removed.

Firstly, I would argue that there are few, if any, users whom the bot has helped avoid misinformation or a skewed perspective. If you know what bias is and how it influences an article then you don't need the bot to tell you. If you don't know or care what bias is then it won't help you.

Secondly, the existence of the bot implies that sources can be reduced to true or false or left or right. Lemmy users tend to deal in absolutes of right or wrong. The world exists in the nuance, in the conflict between differing perspectives. The only way to mitigate misinformation is for people to develop their own skeptical curiosity, and I think the bot is more of a hindrance than a help in this regard.

Thirdly, if it's only misleading 1% of the time then it's doing harm. IDK how sources can be rated when they often vary between articles. It's so reductive that it's misleading.

As regards an open database of bias, it doesn't solve any of the issues listed above.

In summary, we should be trying to promote a healthy sceptical curiosity among users, not trying to tell them how to think.

Who fact-checks the fact-checkers? Fact-checking is an essential tool in fighting the waves of fake news polluting the public discourse. But if that fact-checking is partisan, then it only acerbates the problem of people divided on the basics of a shared reality.

This is why a consortium of fact-checking institutions have joined together to form the International Fact-Checking Network (IFCN), and laid out a code of principles. You can find a list of signatories as well as vetted organizations on their website.

MBFC is not a signatory to the IFCN code of principles. As a partisan organization, it violates the standards that journalists have recognized as essential to restoring trust in the veracity of the news. I've spoken with @Rooki@Lemmy.World about this issue, and his response has been that he will continue to use his tool despite its flaws until something better materializes because the API is free and easy to use. This is like searching for a lost wallet far from where you lost it because the light from the nearby street lamp is better. He is motivated to disregard the harm he is doing to !politics@Lemmy.World, because he doesn't want to pay for the work of actual fact-checkers, and has little regard for the many voices who have spoken out against it in his community.

By giving MBFC another platform to increase its exposure, you are repeating his mistake. Partisan fact-checking sites are worse than no fact-checking at all. Just like how the proliferation of fake news undermines the authority of journalism, the growing popularity of a fact-checking site by a political hack like Dave M. Van Zandt undermines the authority of non-partisan fact-checking institutions in the public consciousness.

To clarify what MBFC considers "MIXED" factual reporting (the same rating they give known disinformation factory Breitbart):

Further, while The Guardian has failed several fact checks, they also produce an incredible amount of content; therefore, most stories are accurate, but the reader must beware, and hence why we assign them a Mixed rating for factual reporting.

They list like five fact checks, while The Guardian puts out basically quintuple that every day. And moreover, this is the sort of asinine nitpick that they classify as a "fact check".

"Private renting is making people ill." "Private renting is making people ill, but maybe this happens with other housing situations too, we don't know, so we rate this as false."

MBFC's ratings for "factual reporting" are a joke.

Remove it.

No need for a bot. Obvious misinformation should be removed by the mods. Bias is too subjective to be adjudicated by the mods. Just drop it already. It's consistently downvoted into oblivion for a reason. The feedback has been petty damn obvious. This whole thread is just because the mods are so sure they're right that they can't listen to the feedback they already got. Just kill the bot.

The bot is basically a spammer saying "THIS ARTICLE SUCKS EVEN THOUGH I DIDN'T READ IT" on every damn post. If that was a normal user account you'd ban it.

This thread is a mess.

users: "bot is awful"

mod: "ok so it's not terrible so that's good"

It has been helpful and we would like to keep it around in one form or another.

Bull fucking shit. The majority of feedback has been negative. I can't recall a single person arguing in its favor, but I can think of many, myself included, arguing against it. I hope you can find my report of one particular egregious example, because Lemmy doesn't let me see a history of things I reported. I recall that MBFC rated a particular source poorly because they dared to use the word "genocide" to describe what's going on in Gaza. Trusting one person, who clearly starts from an American point of view, and has a clearly biased view of world events, to be the arbiter of what is liberal or conservative, or factual or fictional, is actively harmful.

No community, neither reddit nor Lemmy nor any other, has suffered for lack of such a bot. I strongly recommend removing it. Non-credible sources, misinformation, and propaganda are already prohibited under rule 8. If a particular source is so objectionable, it should be blacklisted entirely. And what is and is not acceptable should be determined in concert with the community, not unilaterally.

Edit: And another thing! It's obnoxious for bot comments to count toward the number of comments as shown in the post list. Nobody likes seeing it and thinking "I wonder what people are saying about this" and it's just the damn bot again. But that's really a shortcoming in Lemmy.

In literally every thread I've seen it post in, it gets downvoted to hell.

The bot is basically loud as fuck in a way that disrupts the comment feed.

Imagine how comments should create and add to a conversation. Imagine how various lemmy clients feed or service that conversation….

Now imagine how a double dropdown big as fuck post says “fuck you” to that conversation.

Just please consider how the form of your shit can be just as imposing as the content, which I really appreciate.

Yet somehow your posts always have me thinking “shut the fuck up” which seems antithetical to building a community.

1. Please, move the bias and reliability outside of the first accordion/spoiler. This is the sole purpose the bot was meant to provide. If we can't see that at a glance, it's bad. I don't see how these few words are "too long" either. I feel like a lot of the space could be cleared by turning the "Search Ground News" accordion into another link in the footer.
2. While I personally don't see the point of the controversy, it wouldn't be too hard to manually enter Wikipedia's Perennial Sources list into the database that the bot references, especially with MediaWiki's watchlist RSS feed. This would almost certainly satisfy the community.
3. Open source the database and the bot. Combined with #2, this could also offer an API to query Wikipedia's RSP for everyone to use in the spirit of fedi and decentralization.

I'm gonna be Left-Center on this with reliable credibility that the bot is useless at best.

It is reporting on the source, not the content, of what is posted which is already going to be a problem for discourse.

If there are media sources that are known or proven to be a problem, I would find it preferable the bot just alert that and ignore anything else.

Get rid of it entirely. In another one of your comments you acknowledged that it "seemed" like the bot is an extension of the mods telling everyone else what to think. You are close. It doesn't seem that way, it is that way.

Also, bot is annoying AF. If you really are in love with so much, make it an opt-in service and it can DM all the psychos who want to be spammed by it.

I wish the comment count on a post didn't include MBFC (or maybe bots in general).

I blocked it straight away so I don't have a dog in this fight but I'm instantly skeptical of any organization that claims to be the arbiter of what is biased and to what degree.

@jeffw@lemmy.world Why did you stop replying to posts here? Most people is telling you the bot is bullshit. You stopped commenting in this thread while being active elsewhere, are you going to take action or not?

Remove it please. It's an obtrusive advertisement for Ground News.

It's incredibly annoying to see comments: 1, only to click the post to see an ad. It makes me less inclined to interact with Lemmy at all. It's the same kind of crap that ruined Reddit.

How do you rate bias without bias? What is the bot's definition of left or right? How did you build your ratings?

It's all bullshit, man.

How much more feedback do you need to gather on the subject to understand that a bot with a garbage datasource is no use to anyone? Even opening this thread is an insult and a sign of how little you recognize and care for your community. Remove the shit bot already instead of fishing for excuses to keep it active.

Yes MBFC is Extremely American in its definitions of left and right. A less US-centric rating would be much preferred.

I generally like the idea otherwise.

Just kill it with fire.

Im sorry but the sole concept of the bot is bullshit and as many have said already the idea is biased per se. I wish i lived in the same world as mbfc where it seems like all media is left-center.

If anything, what would be needed would be a bot that checked if the information on that article has any known missinformation or incorrect/wrong facts. And that would be extremely hard to maintain and update as a lot of news are posted before any fact checking can be done.

I think this tool, while probably well-intended, only adds to the polarization problem of the world.

There's no such thing as an objective left or right. It's a relative scale. You shouldn't have a bot calling things left or right at all.

Also don't push Ground News. They already get plenty of press from their astroturfing.

Here's the comment reply from when I first asked what was wrong with MBFC. Gotta say. I agree with that comment. I'm surprised more people haven't posted similar examples here.

https://lemmy.dbzer0.com/comment/12328918

Edit: here is the text from the linked comment.

I'm just gonna drop this here as an example:

https://mediabiasfactcheck.com/the-jerusalem-report/

https://mediabiasfactcheck.com/the-jerusalem-post/

The Jerusalem Report (Owned by Jerusalem Post) and the Jerusalem Post

This biased as shit publication is declared by MBFC as VEEEERY slightly center-right. They make almost no mention of the fact that they cherry pick aspects of the Israel war to highlight, provide only the most favorable context imaginable, yadda yadda. By no stretch of the imagination would these publications be considered unbiased as sources, yet according to MBFC they're near perfect.

I'm frankly rather concerned about the idea of crowdsourcing or voting on "reliability", because - let's be honest here - Lemmy's population can have highly skewed perspectives on what constitutes "accurate", "unbiased", or "reliable" reporting of events. I'm concerned that opening this to influence by users' preconceived notions would result in a reinforced echo chamber, where only sources which already agree with their perspectives are listed as "accurate". It'd effectively turning this into a bias bot rather than a bias fact checking bot.

Aggregating from a number of rigorous, widely-accepted, and outside sources would seem to be a more suitable solution, although I can't comment on how much programming it would take to produce an aggregate result. Perhaps just briefly listing results from a number of fact checkers?

Ban it and all bots honestly. I hate seeing a comment on a thread just to find out it's a bot. If not use like this continues we might see a fresh post with 6 new comments, all of them bots that don't add to the discussion.

We need a bot that will tell us what the bias of the bot is.

Tell the bot to never be the first comment. I find it very frustrating when I see "a comment on this post" and it's just the bot. I'm here to read what people have to say so it is very annoying when I think someone said something and it's just the bot.

There was even a front page meme about this last year, but with another noisy bot. Lemmy doesn't bury new comments like Reddit does, so there's no real penalty to making the bot wait.

I'm not sure what to do here. On my mobile device the compacted media bias fact check post still takes up 50% of my phone screen.

How a post tag if we have a tagging system in Lemmy, instead of a whole long comment?

Maybe the bot could just post a one line summary with a link to more information?

You don’t need to manufacture an authoritative source of truth as you the mods see it.

Just write down what you see as the truth and that you’ll ban anyone who speaks out against it.

Stop trying to build a machine to do the work of creating an echo chamber for you.

A way to improve the MBFC bot would be to delete it.

Failing that, a way to improve the community would be to ban it.

While I think it's important to have some sort of media bias understanding, I dislike the bot being the first (and sometimes only) comment on a post. Maybe it should be reserved only for posts that are garnering attention, and has a definitive media bias answer for (the no results comments are just damn annoying to see).

It also has the knock-on effect of boosting the post higher in whichever sorting algorithm users are using. So it often feels artificially controlled whenever something has 100+ upvotes and less than 10 comments, knowing the first comment is always a bot. Like, would it be fair for me to have 10 bots that comment factual information of posts I personally like, just to boost their visibility?

The bot has no purpose. Either an article can be posted or not there's no reason for the bot prompt. It just looks like thought policing using a bias checker which 'coincidentally' prefers what the current Democrats position is.

I can hardly imagine the bot stopping any fake news from being posted either.

Unfortunately the bot is fatally flawed as long as it's just repeating MBFC information. I would be interested in a community program but I have the same end worry. What's the risk that we create an echo chamber? It might be better than an echo chamber based on MBFC ratings but it's still an issue worth worrying about.

That said I'm down to try a community approach.

I wound up blocking your shitty bot because it spammed pretty much every post.

Get rid of the bot

Probably not an issue with the bot itself, but just FYI, it appears the spoiler tags don't work on Boost.

Addressing the Overton window issue is the main fix I would hope for.

The proposed solution of a home-brewed open-source methodology of determining bias without the Overton influence would be a very welcome improvement in my opinion.

Credibility isn't subjective. It should be a hard value.

Orientation is indeed subjective and unless in the extremes should (imo) not be defined

Although I do see that that bot has a very slight right wing bias I like it. It prevents the normalization of the use of literal propaganda outlets as news sources.

I have a suggestion that might be a good compromise.

The bot only comments on posts that are from less factual news sources or are from extreme ends of the spectrum.

On a post from the AP the bot would just not comment.

On a post from Alex Jones or RT the bot would post a warning.

That way there is less “spam”, but people are made aware when misinformation or propaganda is being pushed.

Also with such a system smaller biasses are less relevant and therefore become less important.

Personally I'm in favor of the bot. One complaint I've seen that I agree with is that it doesn't need to float high up in the comments. If it was simply made to not upvote itself, it would stay nearer to the bottom naturally, which I think would be preferable.

How about just finally making the bot open source and let people comment or contribute there?

Bias ratings will always be biased. So aggregate or having multiple sources briefly used in a single small post would work best.

I hate that I have to expand the section to see the rating. If that could be fixed, it'd be better.

Not directly related to MBFC bot, but what's your opinion on other moderation ideas to improve the nature of the discussion? Something Awful forums have strawmanning as a bannable offense. If someone says X, and you say they said Y which is clearly different from X, you can get a temp ban. It works well enough that they charge a not-tiny amount of money to participate and they've had a thriving community for longer than more existing social media has been alive. They're absolutely ruthless about someone who's being tricksy or pointlessly hostile with their argumentation style simply isn't allowed to participate.

I'm not trying to make more work for the moderators. I recognize that side of it... the whole:

This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.

... makes perfect sense to me. I get the idea of mass-banning sources to get rid of a certain type of bad faith post, and doing it with automation so that it doesn't create more work for the moderators. But to me, things like:

• Blatant strawmanning
• Saying something very specific and factual (e.g. food inflation is 200%) and then making no effort to back it up, just, that's some shit that came into my head and so I felt like saying it and now that I've cluttered up the discussion with it byeeeeee

... create a lot more unpleasantness than just simple rudeness, or posting something from rt.com or whatever so-blatant-that-MBFC-is-useful type propaganda.