GPT-3 generative AI

This new AI platform for generating text, and apparently, code, web sites, and other stuff, has been getting a lot of attention and I have to say, I'm usually underwhelmed by this stuff but the demos and examples I've seen are pretty spooky I have to say.

I bet there are people on here who know about this and are using it and I'd like to hear about it. A friend said an RPG he plays was using it and I've messed around with Shortly which is a short story writing app that uses it. I suspect you could use for music and probably that's already being done somewhere.

The GPT-3 generated Star Trek episode descriptions are pretty hilarious and scary https://twitter.com/startrekai?lang=en

  On 8/6/2020 at 2:59 PM, zkom said:

ZOMG instafollow

  3 minutes ago, zkom said:

 

The GPT-3 generated Star Trek episode descriptions are pretty hilarious and scary https://twitter.com/startrekai?lang=en

 

Expand  

I can see how this sort of tech can easily be abused.

What happens if you feed the source code of GPT 3 into GPT 3?  (or is it only for English language and not for coding language?)

  On 8/6/2020 at 3:29 PM, dingformung said:

What happens if you feed the source code of GPT 3 into GPT 3?  (or is it only for English language and not for coding language?)

It's been basically fed the whole of the text content of internet that's publicly available so it contains tons of source code as material already. You need to just prompt it to start the code generation.

Talking about coding. Check this natural language shell that uses the OpenAI API. jfc

  On 8/6/2020 at 3:43 PM, zkom said:

It's been basically fed the whole of the text content of internet that's publicly available so it contains tons of source code as material already. You need to just prompt it to start the code generation.

So as I understand it you can feed it text - or any info - and it logically completes it/"thinks" it further with the knowledge it has (which is basically all public knowledge on the internet). So what if it is generating source code based on its own source code? Is GPT 3 able to generate GPT 4? Is GPT 4 able to generate GPT 5? Etc.

Sorry, totally ignorant on coding and machine learning.

  On 8/6/2020 at 4:04 PM, dingformung said:

So as I understand it you can feed it text - or any info - and it logically completes it/"thinks" it further with the knowledge it has (which is basically all public knowledge on the internet). So what if it is generating source code based on its own source code? Is GPT 3 able to generate GPT 4? Is GPT 4 able to generate GPT 5? Etc.

Sorry, totally ignorant on coding and machine learning.

No, it can't think or understand what it is writing or what the real world context is. It just basically knows what kind of words go together. If you try to generate code it's syntax might be correct but logically it can be complete nonsense.

For example here's a cookie recipe generated by GPT-3. It doesn't make much sense even though it can kind of keep up with the ingredients to stay consistent, etc. The second step already throws a dog in a ziploc bag.

Would it be possible to have an OS using AI to keep things running and correcting errors that might occur and adapting if new hardware is installed. Or would it be unfeasible and pointless?

  On 8/6/2020 at 3:29 PM, dingformung said:

What happens if you feed the source code of GPT 3 into GPT 3?  (or is it only for English language and not for coding language?)

It can write code too, either automatically by looking at code you've already written and extrapolating out to what the completed code would look like, or by examining some English language description of what the code can do and creating it from that.

  On 8/6/2020 at 4:11 PM, zkom said:

If you try to generate code it's syntax might be correct but logically it can be complete nonsense.

This isn't correct, it's pretty good at generating perfectly functional code, mightn't be exactly what you want, or could have bugs in it, but at this early stage it's very impressive.

  On 8/6/2020 at 5:24 PM, caze said:

This isn't correct, it's pretty good at generating perfectly functional code, mightn't be exactly what you want, or could have bugs in it, but at this early stage it's very impressive.

Let me rephrase a bit, it can write completely functional code, but it doesn't understand what the code does. It doesn't know the meaning of any of the commands, operators, etc. It only knows the models that it has built based on the data that has been used as input. It's impressive what it can do with those models, but it still doesn't know what happens when you run the code and the code is just the best guess it has based on the previous examples it has seen.

  On 8/6/2020 at 4:49 PM, azatoth said:

Would it be possible to have an OS using AI to keep things running and correcting errors that might occur and adapting if new hardware is installed. Or would it be unfeasible and pointless?

This would probably eventually lead to a system that nobody understands how or why it works and if something goes wrong that the AI can't fix then it's unfixable.

It's certainly good enough to generate WATMM responses. Would be fun to have a WATMM AI bot as a mascot.

Or how about an online forum with only AI bots that respond to each other? Would be sort of pointless

Is it possible to create a recursive feedback loop of a code-generating AI recreating new versions of itself infinitely based on its own source code, given there is infinite processing power. 

???WHAT IF WATMM IS A BOT AND YOU ARE THE ONLY REAL USER? ??....

  On 8/6/2020 at 7:01 PM, zkom said:

Let me rephrase a bit, it can write completely functional code, but it doesn't understand what the code does. It doesn't know the meaning of any of the commands, operators, etc. It only knows the models that it has built based on the data that has been used as input. It's impressive what it can do with those models, but it still doesn't know what happens when you run the code and the code is just the best guess it has based on the previous examples it has seen.

Yes, that's correct. It has no long-term memory (it does have a short term memory though, it can remember stuff in the context of the thing it's trying to generate, but once the calculation is complete that memory goes away), no abstract reasoning ability, etc.

  On 8/6/2020 at 4:20 PM, zkom said:

For example here's a cookie recipe generated by GPT-3. It doesn't make much sense even though it can kind of keep up with the ingredients to stay consistent, etc. The second step already throws a dog in a ziploc bag.

 

will try this tonight and report back

  On 8/6/2020 at 7:21 PM, caze said:

Yes, that's correct. It has no long-term memory (it does have a short term memory though, it can remember stuff in the context of the thing it's trying to generate, but once the calculation is complete that memory goes away), no abstract reasoning ability, etc.

That's what it wants you to think.

  On 8/6/2020 at 7:19 PM, dingformung said:

???WHAT IF WATMM IS A BOT AND YOU ARE THE ONLY REAL USER? ??....

I was totally unimpressed with modern AI tech until i saw 20 chess games between AlphaZero (neural network based chess program) and Stockfish (one of the strongest chess engines) released in 2017 by Google's DeepMind (company that developed AlphaZero); soon they released 100 additional games. Developers gave AlphaZero only basic rules of chess and they left it to play with itself. After only 4 hrs of playing/teaching it achieved rating higher than the strongest supercomputer program and after 24 hrs a superhuman level of playing nobody believed it'd be possible in the near future.

 But the games was totally mind blowing! I've never seen that level of sophistication, neither in human or supercomputer playing, sacrificing pieces for no apparent reasons, crazy moves and yet it wins in totally unsuspecting ways. AI is the closed we can can to experience an alien mind imo; cause it feels alien... it's just otherworldly! 

if someone's interested in chess and this topic, here's an example: 

ok... one more

On the one hand it's really impressive.

On the other hand it all feels ... alien, like others have said.

Also a bit of an empty victory. It seems somewhat disappointing that investing all this time and effort results into something that insists on adding a dog to chocolate chip cookie dough.

Which brings me to:

The resources used. Seriously colossal amounts of energy were expended on training this thing. This sort of machine learning really does not scale. If we keep doing it this way, sooner or later we will need more computing power for training our models than there are atoms in the universe.

It's starting to feel that machine learning for AI may be something of a dead end. Useful for a lot of things, but if we really want to make cars drive themselves without them being steered off course by stickers put on roads, or if we want computer vision systems to be able to accurately distinguish between a fish and a bicycle, automating statistics may not be the (only) way to go.

How our own subconscious works is also pretty alien to us, the current level of AI is somewhat similar to our subconscious processing in that regard, albeit just a limited type of it, once we have a bunch of different types of things, and start to wire them up to one another in interesting ways, then things will start to get really interesting.

  On 8/6/2020 at 7:19 PM, dingformung said:

Or how about an online forum with only AI bots that respond to each other? Would be sort of pointless

 

https://www.reddit.com/r/SubSimulatorGPT2

dunno, as soon as it is able to mutate by its own it might do interesting stuff at some point. doesn't matter whether all of it is useful or not. can just be inspiring/funny/interesting. sort of like technological poetry.

  On 8/6/2020 at 10:16 PM, caze said:

How our own subconscious works is also pretty alien to us, the current level of AI is somewhat similar to our subconscious processing in that regard, albeit just a limited type of it, once we have a bunch of different types of things, and start to wire them up to one another in interesting ways, then things will start to get really interesting

you mean using brain computer interfaces that are wired to neurons? ... and that can be used to (indirectly) wire neurons to other neurons that don't belong to the same brain? the indirectliness can vary of course and be minimal

  On 8/6/2020 at 10:26 PM, zazen said:

https://www.reddit.com/r/SubSimulatorGPT2

It imitates Trump perfectly

  Quote

"We have a lot of people, a lot of people, a lot of people that are very angry, and they're very, very, very -- I mean, you have many people and I can't even count them all, I can't even list them off, I can't even, I can't even -- I'm -- I can't even -- you know, I mean there's many people that are very angry, and they're very, very, very, very, very, very angry, and they're -- they're being very misused."