My editor-in-chief dared me to tug a bunch of random subjects out of a hat and write a compelling tech story about them. Right here’s what I bought: Jay-Z, neural networks, the US Navy SEALs, and jazz music.
Fortunately for me, I occurred throughout this video that includes a gaggle of proficient jazz musicians deciphering a Deepfake AI-generated model of Jay-Z rapping the infamous Navy SEAL copypasta meme. Natch.
Adam Neely, a well-liked music YouTuber with over 1,000,000 subscribers, launched the video as a part of a CuriosityStream documentary. It contains a very human quest to interpret a computer-generated rap acapella.
Jay-Z is a implausible goal for a Deepfakes AI as a result of, not like most rappers, he’s launched quite a lot of official, studio high quality acapella tracks. This enables AI builders to coach a neural community on his voice and rapping type with out worrying about eradicating music or extracting vocals.
In principle, this could imply we are able to “type switch” Jay’s stream, however that’s not fairly how issues work out. As Neely explains within the video, there’s an “uncanny valley” for music identical to there may be for visible imagery. The best way this works for music is that, in some unspecified time in the future, pc generated vocals begin getting form of cute. Suppose “Nonetheless Alive” by GLaDOS:
However the nearer you get to dialing in a “actual” human voice, the weirder it begins to get. It’s not fairly so noticeable while you’re watching a Deepfake the place President Barrack Obama gives a speech, however while you actually dig into the best way issues sound – like say while you’re making an attempt to make novel music – the issues within the system begin to present.
For starters, Neely needed to name on skilled jazz musicians to assist him make heads or tails of the faux-Z vocal tracks. Within the video, he says:
It didn’t have any of the musical features that we’d usually discover in hip hop music, specifically a gradual pulse and any form of rhyme or rhythmic schemes. However there’s something form of musical there or rhythmic about it. At the very least that’s how I’m listening to it.
A part of the difficulty is that the pretend Jay-Z is utilizing a copypasta as lyrics as a substitute of one thing written to a beat. Because of this it doesn’t comply with any regular sample of talking and that makes it extremely exhausting to transpose into music. In different phrases: the present technology of Deepfakes are unlikely to provide a brand new Tupac or Biggie Smalls album anytime quickly. They will copy the voice nicely sufficient for parody, however the musicianship isn’t fairly there but.
Neely wasn’t going to be derailed by the truth that robots suck at rapping. When he turned to his associates within the jazz music enterprise, they have been in a position to flip the scat-singing-like Deepfake utterances into as compelling a jazz music that includes an AI-generated Jay-Z as might be humanly doable.
Take a look at the total video above for particulars.
Printed August 4, 2020 — 18:08 UTC