spectral mapping

i was just rewatching the mtv interview (with him sitting in all the donkey rhubarb donkeys) and he mentioned that for tracks like milkman he used "spectral mapping" to take a famous persons voice and map it over his intonation/the words he was saying to alter his voice and make it sound better. my question is: does anyone know what this is? my first guess would be that it involves spectral analysis (via fourier transformations) to examine which harmonics are prevalent in a given persons voice and then somehow applying them to his own voice? i'm curious as to how he could do this without losing the actual words he was saying (mouthing/warping of sounds that we use to shape words is what im getting at). any ideas?

edit: mods, perhaps this is better off in ekt? not sure... up to you.

Wow very interesting. You have any links?

Probably a similar process to vocoding, if not exactly the same. Just using the select harmonics of that persons voice.

My guess is it isn't some elaborate magical process, just something similar to vocodering or autotune that he described in a way that made it sound all high-tech.

Wasn't it all about some specific modulation ?

If anything could do it Kyma could

  On 2/10/2010 at 9:42 AM, Mesh Gear Fox said:

probably a little fib but I can imagine it would be convolution of your own speech and a sort of "sound profile" or another persons voice. I guess it's possible but no idea how well it would sound.

this sounds most likely, as convolution seems like the perfect operation (at least from my math background) for the job. i wonder how one constructs such a profile without being too dependent on the specific words/intonation being used in a sample. perhaps its created with a statistical analysis.

  On 2/10/2010 at 4:39 PM, futureimage said:

Probably a similar process to vocoding, if not exactly the same. Just using the select harmonics of that persons voice.

that was my first thought, but i feel like vocoding with any standard waveforms would create too "pure" a sound to be human. perhaps this is done in a matter similar to the convolution earlier?

  On 2/10/2010 at 5:52 PM, marf said:

If anything could do it Kyma could

yeah! he's basically describing what the kyma folks call "tau synthesis" (a term they invented, as it's hard to find any literature -- even in journals -- that is in the field of dsp. most of it's neuroscience). the main reason i didn't follow this path was i was fairly sure this predated kyma, etc. man, if they weren't quite so cost prohibitive i'd be all over them.

i'm digging around symbolic sound's web page to see if they have any documents on it, as their samples with tau synthesis -- both for morphing between sounds and layering them in the manner i've described -- sound spot on. essentially what i'm getting at is the point halfway between a timbre morph. this is an effect i've yearned to understand for a while, so let's do it! the idea of a spectral mapping seems legitimate, even though the term sounds bogus at first. it suggests looking at the frequency domain and trying to map it over to some other input. when i get home i'll try convolving various signals to see what i get.

note: i've just sent an email to their info center requesting any literature they have on the subject!

edit: lol, i got so carried away in my email i ended up asking if they'd be interested in hiring me as an intern... i'll keep you all posted!

I think what it does is you analyze a voice, get the harmonic content and then take your voice and extract the transients (the spits, chiffs and mouth sounds) and convolve them together. Im sure it doesn't sound completely natural

  On 2/10/2010 at 8:21 PM, marf said:

I think what it does is you analyze a voice, get the harmonic content and then take your voice and extract the transients (the spits, chiffs and mouth sounds) and convolve them together. Im sure it doesn't sound completely natural

this seems to be honing in a bit -- yeah it certainly doesn't sound natural. i suppose i'm more concerned with the algorithmic process by which that content is extracted. i imagine spectral analysis comes in handy for almost generating a "key" in which various voices are played (low voices, mid voices with high overtones, etc).

rich was into a prog called lemure pro.

http://www.partnersinrhyme.com/mac/Music.shtml

Also Cdp which has spectral algs

I think he talked bullshit.

  On 2/10/2010 at 11:24 PM, chimera slot mom said:

I think he talked bullshit.

whether the name of the process is bs or not, the process itself is very real! my search has lead me to the "McAulay-Quatieri" analysis algorithm, which is an analogue of the fast fourier transform. digging deeper!

  On 2/10/2010 at 5:52 PM, marf said:

If anything could do it Kyma could

we have a winrar.

I have and use Kyma X. One of the immense features is real time morphing which isnt usually possible with other programs. This is how it works, ill use the example they give in the manual.

You have a recording of a lion, kyma analyses the harmonic content and how it changes during the recording, then it analyses the volume throughout the recording. Now if you run your voice through a microphone, the recording loops and kyma applies the changing harmonic content ( which then follows the fundamental pitch of your voice so you can change the pitch ) and applies the volume envelope to it.

Ive done a couple of attempts at it and its really hard to get right, but when you do its well good.

This is actually quite amazing - the implications for movie soundtrack work obviously spring most immediately to mind, that you could overlay actors' voices with the resonance of almost anything.

But it still begs the question, so if RDJ *did* do this as he claimed back in '96 or whenever, how come all the tracks of him singing from that period still sound so much like him? ha ha. (Not that I have a problem with that - I do actually think he's got a lovely quality to his voice - but still. It does make me think it was just another little fib - even if the concept is indeed sound and the technology exists to make it possible.)

Maybe he just used it a little bit. Or he did some takes with the full effect & decided the dry version sounded better.

Actually, I dunno. Tracks like To Cure A Weakling Child he's obviously fucked with his own voice so much that he's clearly used some treatment on it, maybe that's where it ended up.

Or maybe - DELIBERATELY CONTENTIOUS ALERT - he used it on Beetles to needlessly provoke a ridiculous argument for years to come by shifting his consonant sounds into ranges that only certain people can hear so that a perfectly clear word will sound totally indistinct and like something else to those whose hearing extends into that range. Hrrmmmmm? Wouldn't put it past him.

  On 2/11/2010 at 1:28 PM, Masonic Boom said:

This is actually quite amazing - the implications for movie soundtrack work obviously spring most immediately to mind, that you could overlay actors' voices with the resonance of almost anything.

 

But it still begs the question, so if RDJ *did* do this as he claimed back in '96 or whenever, how come all the tracks of him singing from that period still sound so much like him? ha ha. (Not that I have a problem with that - I do actually think he's got a lovely quality to his voice - but still. It does make me think it was just another little fib - even if the concept is indeed sound and the technology exists to make it possible.)

yeah... tracks like windowlicker sound like they could have been done with a sampler. and tracks like love 7. hmm... well i'm still digging to figure out what exactly kyma does. their info people seemed to kind of dismiss my question and link me to their product page.

the closest you hear of stuff like this is cassette boy. Its all cut up work though

You can do stuff like this with metasynth.

http://www.uisoftware.com/MetaSynth/index.php

metasynth does harmonic resynth, but i don't know if you can use it for this technique (perhaps you could use the new harmonic synth in a vocoder with the target vocal track!).

yeah im pretty sure most of the spectrally sounding stuff on that era of Aphex work is done on metasynth, he was really pushing that program and figuring out cool ways to use it.

I own a kyma and i don't recognize any obvious uses of it, Funny little man comes to mind as a possibility but i think its mroe likely he's just using an Eventide Eclipse or kind of later period Harmonizer that has the auto-tune feature on it.

In what way is Weakling Child not him? PYou can hear his distinctive voice on the bits where it's processed the least and he's singing most clearly - especially on the "ears" bit. I'm willing to accept that there's pitch shifting and bit of processing but to my ears it sounds like every other known example of his speech.

To me it sounds like it could be RDJ falsetto cleaned up a bit on the computer, maybe with a little additional pitch shifting.

I'd say that RDJ falsetto singing is one of my favourite sounds in the universe, but I'll be accused of talking about his genitals again.

There's some ring modulation going on as well as pitch shifting, to produce that kind of metallic tone.