Why AI detectors think the US Constitution was written by AI
Can AI writing detectors be trusted? We dig into the theory behind them.
As expected, they can't be trusted. And the more AI evolves, the less likely AI content will be detectable IMO.
It will almost always be detectable if you just read what is written. Especially for academic work. It doesn't know what a citation is, only what one looks like and where they appear. It can't summarise a paper accurately. It's easy to force laughably bad output by just asking the right sort of question.
The simplest approach for setting homework is to give them the LLM output and get them to check it for errors and omissions. LLMs can't critique their own work and students probably learn more from chasing down errors than filling a blank sheet of paper for the sake of it.
LLMs can't critique their own work
In many cases they can. This is commonly used to improve their performance: https://arxiv.org/abs/2303.11366
I think there’s a big difference between being able to identify an AI by talking to it and being able to identify something written by an AI, especially if a human has looked over it for obvious errors.
What you are describing is true of older LLMs. GPT4, it's less true of. GPT5 or whatever it is they are training now will likely begin to shed these issues.
The shocking thing that we discovered that lead to all of this is that this sort of LLM continues to scale in capabilities with the quality and size of the training set. AI researchers were convinced that this was not possible until GPT proved that it was.
So the idea that you can look at the limitations of the current generation of LLM and make blanket statements about the limitations of all future generations is demonstrably flawed.
I'm no GPT booster, but I think that the real problem with detectability here
It will almost always be detectable if you just read what is written. Especially for academic work.
is that it requires you to know the subject and content already, and to be giving the paper a relatively detailed reading. For a rube reading the paper, trying to learn from it - a lot of GPT content is easily mistaken as legitimate. And it's getting better. We're not safe simply assuming that AI today is as good as it will ever get and the clear errors we can detect cannot ever be addressed.
Penetrating academic writing, for academics, is probably one of the highest barriers of any writing task, AI or not.
But being dismissive of the threat of AI content because it's not able to convincingly fake some of the hardest writing that real people do is maybe sidestepping a lot of much more casual writing - that still carries significance and consequence.
Chad comment right here...
Clearly the Founding Fathers were not advanced enough to have crafted the US Constitution unaided. It's only reasonable to imagine that ancient aliens could have landed, given them an AI to assist them, and then departed with nobody the wiser.
I am certain we can find evidence of this if we dig hard enough.
James Madison was a fraud! He used ChatGPT instead of writing out his homework like he was supposed to. Smh my head.
Clearly the Founding Fathers were not advanced enough to have crafted the US Constitution unaided.
In a sense you are correct. They cribbed from lots of the most well known political philosophers at the time. For example, there are direct quotes from Locke in the Declaration and his influence over the Constitution can be felt clearly.
I am certain we can find Georgios Tsoukalos saying there is evidence of this sooner or later. Probably extraterrestrial!
As an AI language model, I am unable to found a country but here are some ideas which might help people feel less likely to revolt.
This is a simple example of a constitution. For a real constitution you might want to consider a more detailed list of amendaments and civil rights. Please refer to a professional constitutional writer if you want to create a constitution for productive purposes
GPT-4 is quite willing to help you create the constitution for your own country, well I guess I have to find a name now:
Certainly! Crafting a constitution for a new country is an exciting and monumental task. The constitution is the core document that dictates the legal obligations and rights of your country's government and its citizens. Here's a very basic outline to get us started. Please remember, this is a rudimentary draft and it will need to be adjusted based on the specific needs, culture, history, and goals of your country:
PREAMBLE
We, the people of [Your Country's Name], in order to form a more perfect union, establish justice, ensure domestic tranquility, provide for the common defense, promote the general welfare, and secure the blessings of liberty to ourselves and our posterity, do ordain and establish this Constitution for [Your Country's Name].
ARTICLE I: THE LEGISLATURE
Section 1: All legislative powers herein granted shall be vested in a [name of your legislative body].
Section 2: [Details on how members are elected, their terms, qualifications, powers and responsibilities]
Section 3: [Details on how legislation is passed, veto powers, etc.]
ARTICLE II: THE EXECUTIVE
Section 1: The executive power shall be vested in a [title of your head of state].
Section 2: [Details on how the head of state is elected, their term, qualifications, powers and responsibilities]
Section 3: [Details on how the executive branch operates, and other positions within the executive branch]
ARTICLE III: THE JUDICIARY
Section 1: The judicial power of [Your Country's Name] shall be vested in one Supreme Court, and in such inferior courts as the [Legislative body] may from time to time ordain and establish.
Section 2: [Details on how judges are appointed, their terms, qualifications, powers and responsibilities]
ARTICLE IV: THE RIGHTS OF THE PEOPLE
Section 1: All persons born or naturalized in [Your Country's Name] are citizens of [Your Country's Name] and of the state wherein they reside. No state shall make or enforce any law which shall abridge the privileges or immunities of citizens.
Section 2: [Further details on the rights, freedoms, and responsibilities of the citizens, these could include things like freedom of speech, freedom of religion, right to a fair trial, etc.]
ARTICLE V: AMENDMENTS
Section 1: The [Legislative body] whenever two thirds of both Houses deem it necessary, shall propose Amendments to this Constitution, or, on the Application of the Legislatures of two thirds of the several states, shall call a Convention for proposing Amendments...
ARTICLE VI: GENERAL PROVISIONS
Section 1: [Any other items that need to be addressed, such as the official language, the national religion (or lack thereof), the national emblem, the process for international treaties, etc.]
ARTICLE VII: RATIFICATION
This Constitution shall be considered ratified when agreed upon by a majority of citizens through a free and fair vote.
Please remember that a constitution is a living document that will grow and change over time as your country evolves. This outline is just a starting point and may not include all the specific sections you need. For example, you might want to add sections addressing human rights, environmental protection, financial regulations, and so on.
This process should involve broad input from many different stakeholders, including legal experts, and should be reflective of the country's culture, values, history, and aspirations for the future. Also, please note that the effective implementation of a constitution requires a strong rule of law and mechanisms for holding those in power accountable.
💰
Because AI detectors suck and are the modern day equivalent of dowsing rods?
They're circular. If the text is too predictable it was written by an LLM* but LLMs are designed to regurgitate the next word most commonly used by humans in any given context.
*AI is a complete misnomer for the hi-tech magic 8ball
This article was written to keep people as long on the page as possible. It didn't get to the point before i left. Someone has a tl;dr?
Constitution is a text that appears many times on the internet. ChatGPT's training set probably has multiple copies of it. So it's likely ChatGPT will generate it. Therefore, the detectors are likely to flag it as AI-generated. That's what I got from it, but I also found it difficult to parse. Maybe someone can correct me on this.
I've recently checked my years-old essay using one of these AI plagiarism detectors and it said that the essay was 90% AI written. So either it's all bs or I'm a time travelling AI.
I’m convinced that it’s been trained on top of the essays of middle and high school students that have gone their whole lives without proper education on vocabulary, grammar, and the like. So when asked to evaluate something written properly, it’s flagged as AI.
Garbage in, garbage out. Same as is ever was.
CAUGHT!
Easy fix: run a constitution detector before the AI text detector
Interesting fact: The same parent company owns both Reddit and Turnitin.
I’m waiting for new conspiracy theories after that article hahah.
Obviously the US Constitution was written by AI, we're living in a simulation. Wake up sheeple, the Matrix is real!
Don’t mistake artificial intelligence for actual intelligence.
Aliens.jpg
Awwww. Beat me to it.
This. Changes. Everything.
They only know what they have been fed.
What more likely first/base feeding than the US Constitution's declarations and it's amendments?
Plot twist: it was! :)