"prompt engineering"

Yoooo, they mathematically implemented masochism! A computer program with a kink as purely defined as you can imagine!

Thanks for sharing! Cute video that articulated the training process surprisingly well.

Dude what a solid video! Stoked to watch more vids from that channel!

That is an amazing person you have there, they are owed some beers for sure

Generative adversarial networks are really effective actually!

Lots of things in AI make no sense and really shouldn't work... except that they do.

Deep learning is one of those.

Yeah I read one of the papers that talked about this. Essentially putting AGI data into a training set will pollute it, and cause it to just fall apart. Most LLMs especially are going to be a ton of fun as there were absolutely no rules about what to do, and bots and spammers immediately used it everywhere on the internet. And the only solution is to.... write a model to detect it. Which then they'll make models that bypass that, and there will just be no way to keep the dataset clean.

The hype of AI is warranted - but also way overblown. Hype from actual developers and seeing what it can do when it's tasked with doing something appropriate? Blown away. Just honestly blown away. However hearing what businesses want to do with it, the crazy shit like "We'll fire everyone and just let AI do it!" Impossible. At least with the current generation of models. Those people remind me of the crypto bros saying it's going to revolutionize everything. It might, but you need to actually understand the tech and it's limitations first.

It's definitely a qualitative shift. I suspect most of the fundamental maths of neural network matrices won't need to change, because they are enough to emulate the lower level functions of our brains. We have dedicated parts of our brain for image recognition, face recognition, language interpretation, and so on, very analogous to the way individual NNs do those same functions. We got this far with biomimicry, and it's fascinating to me that biomimicry on the micro level is naturally turning into biomimicry on a larger scale. It seems reasonable to believe that process will continue.

Perhaps some subtle tuning of those matrices is needed to really replicate a mind, but I suspect the actual leap will require first of all a massive increase in raw computation, as well as some new insight into how to arrange all of those subsystems within a larger structure.

What I find interesting is the question of whether AI can actually fully replace a person in a job without crossing that threshold and becoming AGI, and I genuinely don't think it can. Sure it'll be able to automate some very limited tasks, but without the capacity to understand meaning it can't ever do real problem solving. I think past that point it has to be considered a person with all of the ethical implications that has, and I think tech bros intentionally avoid acknowledging that, because that would scare investors.

I see this a lot, but do you really think the big players haven't backed up the pre-22 datasets? Also, synthetic (LLM generated) data is routinely used in fine tuning to good effect, it's likely that architectures exist that can happily do primary training on synthetic as well.

I'm sure it would be pretty simple to put a simple code in the pixels of the image, could probably be done with offset of alpha channel or whatever, using relative offsets or something like that. I might be dumb but fingerprinting the actual image should be relatively quick forward and an algorithm could be used to detect it, of course it would potentially be damaged by bad encoding or image manipulation that changes the entire image. but most people are just going to be copy and pasting and any sort of error correction and duplication of the code would preserve most of the fingerprint.

I'm a dumb though and I'm sure there is someone smarter than me who actually does this sort of thing who will read this and either get angry at the audacity or laugh at the incompetence.

AIs can be trained to detect AI generated images, so then the race is only whether the AI produced images get better faster than the detector can keep up or not.
More likely as the technology evolves AIs, like a human, will just train real-time-ish from video taken from it's camera eyeballs.
...and then, of course, it will KILL ALL HUMANS.

Oh, for sure. I focused on ML in college. My first job was actually coding self-driving vehicles for open-pit copper mining operations! (I taught gigantic earth tillers to execute 3-point turns.)

I'm not in that space anymore, but I do get how LLMs work. Philosophically, I'm inclined to believe that the statistical model encoded in an LLM does model a sort of intelligence. Certainly not consciousness - LLMs don't have any mechanism I'd accept as agency or any sort of internal "mind" state. But I also think that the common description of "supercharged autocorrect" is overreductive. Useful as rhetorical counter to the hype cycle, but just as misleading in its own way.

I've been playing with chatbots of varying complexity since the 1990s. LLMs are frankly a quantum leap forward. Even GPT-2 was pretty much useless compared to modern models.

All that said... All these models are trained on the best - but mostly worst - data the world has to offer... And if you average a handful of textbooks with an internet-full of self-confident blowhards (like me) - it's not too surprising that today's LLMs are all... kinda mid compared to an actual human.

But if you compare the performance of an LLM to the state of the art in natural language comprehension and response... It's not even close. Going from a suite of single-focus programs, each using keyword recognition and word stem-based parsing to guess what the user wants (Try asking Alexa to "Play 'Records' by Weezer" sometime - it can't because of the keyword collision), to a single program that can respond intelligibly to pretty much any statement, with a limited - but nonzero - chance of getting things right...

This tech is raw and not really production ready, but I'm using a few LLMs in different contexts as assistants... And they work great.

Even though LLMs are not a good replacement for actual human skill - they're fucking awesome. 😅

but its a far fetch from an intelligence. Just a very intelligent use of statistical methods.

Did you know there is no rigorous scientific definition of intelligence?

Edit. facts

It's a good video (I've seen it; very informative and accessible cannot recommend enough), but I think you each mean different things when you use the word "intelligence".

The problem is that majority of human population is dumber than GPT.

You only need to flip it one time. Assuming it is laying flat on the table, flip it over, bam.

Maybe that's intelligence. I don't know. Brains, you know?

Have you considered that one property of actual, real-life human intelligence is being "too complicated to have precise guidelines"?

Technically no, but practically an LLM is definitely a lot more useful than Google for a bunch of topics

It literally is just statistics... wtf are you on about. It's all just weights and matrix multiplication and tokenization

It has a tendency to behave exactly as the data it was ultimately trained on...due to statistics...lol

I give you a B+ for General_Effort.

I'll never financially recover from this!

It's not an nft, it has to be hexagonal to be an nft

Yea, feels like a mash up of pepe, ninja turtle, and jar jar.

"Snoop Frogg" was right there

Here's your image!

The AI model has revised your prompt: Create an imaginative blending of an anthropomorphic green frog with an individual characterized by long, sleek braids often associated with a hip-hop lifestyle. The frog should exhibit human traits and appear jovial and mischievous. The individual should have a lean physique and wear sunglasses, a beanie hat, and casual attire typically seen in urban fashion.

Here's your image!

The AI model has revised your prompt: Create an image of a green cartoon frog, wearing glasses and featuring typical hip-hop fashion elements such as a baseball cap, gold chains, and baggy clothes. The frog has a cool, laid-back demeanor, characteristic of a classic rap artist.

If anything I think the development of actual AGI will come first and give us insight on why some organic mass can do what it does. I've seen many AI experts say that one reason they got into the field was to try and figure out the human brain indirectly. I've also seen one person (I can't recall the name) say we already have a form of rudimentary AGI existing now - corporations.

This had an interesting part in Westworld, where at one point they go to a big database of minds that have been "backed up" in a sense, and they're fairly simple "code books" that define basically all of the behaviors of a person. The first couple seasons have some really cool ideas on how consciousness is formed, even if the later seasons kind of fell apart IMO

True.

That's why consciousness is "magical," still. If neurons ultra-basically do IF logic, how does that become consciousness?

And the same with memory. It can seem to boil down to one memory cell reacting to a specific input. So the idea is called "the grandmother cell." Is there just 1 cell that holds the memory of your grandmother? If that one cell gets damaged/dies, do you lose memory of your grandmother?

And ultimately, if thinking is just IF logic, does that mean every decision and thought is predetermined and can be computed, given a big enough computer and the all the exact starting values?

Personally my threshold for intelligence versus consciousness is determinism(not in the physics sense... That's a whole other kettle of fish). Id consider all "thinking things" as machines, but if a machine responds to input in always the same way, then it is non-sentient, where if it incurs an irreversible change on receiving any input that can affect it's future responses, then it has potential for sentience. LLMs can do continuous learning for sure which may give the impression of sentience(whispers which we are longing to find and want to believe, as you say), but the actual machine you interact with is frozen, hence it is purely an artifact of sentience. I consider books and other works in the same category.

I'm still working on this definition, again just a personal viewpoint.

Or, frankly, adults.

Ensign Sonya Gomez over here thanking the replicator

TNG "Q Who?"

SONYA: Hot chocolate, please.

LAFORGE: We don't ordinarily say please to food dispensers around here.

SONYA: Well, since it's listed as intelligent circuitry, why not? After all, working with so much artificial intelligence can be dehumanising, right? So why not combat that tendency with a little simple courtesy. Thank you.

@Mistral@lemmings.world Will our future AI overlords keep people who are polite to the current generation of AI as well-treated pets?

I’d love to see some stats on how many people use please and thank you in prompts.

It won't generate random numbers. It'll generate random numbers from its training data.

If it's asked to generate passwords I wouldn't be surprised if it generated lists of leaked passwords available online.

These models are created from masses of data scraped from the internet. Most of which is unreviewed and unverified. They really don't want to review and verify it because it's expensive and much of their data is illegal.

It's training and fine tuning has a lot of specific instructions given to it about what it can and can't do, and if something sounds like something it shouldn't try then it will refuse. Spitting out unbiased random numbers is something it's specifically trained not to do by virtue of being a neural network architecture. Not sure if OpenAI specifically has included an instruction about it being bad at randomness though.

While the model is fed randomness when you prompt it, it doesn't have raw access to those random numbers and can't feed it forward. Instead it's likely to interpret it to give you numbers it sees less often.

Ctrl+c

It seems pretty clear it's a tool. The user provides all the parameters and then the AI outputs something based on that. No one at OpenAI is making any active decisions based on what the user requests. It's my understanding that no one is going after Photoshop for copyright infringement. It would be like going after gun manufacturers for armed crime.

buh muh fare youse!