Here Come the AI Deepfake Tracks...

The Guardian has an article on OpenAI's latest accomplishment - getting Frank Sinatra to sing a song he never sang:

Crazy! So cool though. 

Thicc Deepsong. Is there any way we can feed that beast ourselves?

Can somebody more savvy explain how these waveforms are made? Is it sampling a gargantuan library and connecting the dots? Is anything synthesized? 

  On 11/9/2020 at 8:59 PM, chim said:

Can somebody more savvy explain how these waveforms are made? Is it sampling a gargantuan library and connecting the dots? Is anything synthesized? 

is it something like the AI is using trillions of sample points, and adapting that voiceprint to the existing track’s original melody and vocal cadence, like autotune?

  On 11/9/2020 at 8:59 PM, chim said:

Can somebody more savvy explain how these waveforms are made? Is it sampling a gargantuan library and connecting the dots? Is anything synthesized? 

  On 11/9/2020 at 8:49 PM, Psychotronic said:

Thicc Deepsong. Is there any way we can feed that beast ourselves?

Yes. 

Colab notebook:
https://colab.research.google.com/github/openai/jukebox/blob/master/jukebox/Interacting_with_Jukebox.ipynb
You can upload any audio in the Colab notebook and use it as the seed for the neural network for generating new audio

Github:
https://github.com/openai/jukebox

Also DataBots have a few papers on their method with more tech details about how they create model inputs and how they train:

https://dadabots.com/science.php

DadaBots did a Frank Sinatra sings Britney Spears 'Toxic' wow

https://futurism.com/mind-melting-ai-frank-sinatra-toxic-britney-spears