Isolating Vocals and other Instruments - An Idea

Hey music makers, I was thinking tonight about the age-old problem of isolating vocal samples from a track for remixing, and I had a programming idea. If there is already a better way to do this, or if this seems like a silly idea, please let me know. I don't want to waste my time programming something that already exists or that won't work.

Here's the idea: Over the course of a sample, every instrument (in this idealized song) has a unique volume change pattern. That is, if you had the instruments already isolated, each would have a uniquely shaped envelope.

Now a primitive method for isolating an instrument is with highpass and lowpass filters. So if instruments A, B and C sounded relatively low, medium and high pitched, then you could make A and C quiet by throwing away most of their (low and high frequency) sound. With this though, you've still got some of the sound of A and C, and you've thrown away parts of B.

But now B is dominant. So my thought was this: Erase every sound that's too quiet. Now you have a very rough approximation of that true envelope of B. It's missing the attacks and releases, and it's smaller (quieter). We'll put the attacks and releases back in last.

So we've thrown away most of A and C and much of B. At this point our sound is tinny. Then we killed the rest of A and C and possibly lost the attacks and releases of B. Now, to recover the high and low parts of B, we have a little bit of code.

The code would do this:

-Run through every frequency and have an associated narrow range. It would start with, say 40-120Hz, then do 80-160Hz, something like that.

-In each range, it would find this dominant envelope. It would then do its best to isolate secondary envelopes.

-After this is completed, the envelopes are compared. Those sounds with approximately the same shape are combined. Each combination would hopefully be an instrument.

As for restoring the attacks and releases, here's the best I've thought of: the software has to make an educated guess. The assumption is that instruments with a short duration have a faster attack and release, and that the slope of the attack (and release) after the cutoff is the same as that before the cutoff. So for each instrument, the software would guess initial attack and release additions to the envelope. Then it would check. For a given instant, say the first attack of a snare, it would look at each frequency which contributed to the snare one at a time. It would then check to see if that frequency approximately fits the curve. If the frequency is over the whole time, it's ok - there's another instrument also there as expected. If it's under then our attack was too slow.

Finally, if instruments A and B have overlapping attacks/releases, the software re-adjusts its attack/release guesses so that the attack/release envelopes add up right. This would also be a tunable parameter, one of those knobs whose name is unhelpful but which does cool things and eventually gets you the right sound.

So I don't know if this would work. If so, I don't know if it's already been done. Also I don't know if there's a better way. I'd appreciate your thoughts. And if it looks hopeful, I'll program it probably this Summer, but anyone else can feel free to go right ahead if you have the urge.

the idea is to simply isolate the spectrum of the voice as mutch as you can, and use its frequencies as a modulation source for the effects you will need in your patch

  jubes said:

the idea is to simply isolate the spectrum of the voice as mutch as you can, and use its frequencies as a modulation source for the effects you will need in your patch

Thanks for simplifying. Didn't know the vocab.

make it

yeah.. you should... like... totally do that.

i dont know what you just said but i like it

i've always had an idea for lifting vocals from tracks, but i've never really researched it or tried it because i've never actually had the need to do it for my own music, but conceptually it seems like it would be possible. basically, you'd be using phase frequency cancellation. it would only work if you had an instrumental version of the track, of course, but in theory, wouldn't you be able to use the instrumental track, combine it with the full track at a slight phase to cancel out all the like frequencies and be left with just the clean vocal track?

  catsonearth said:

i've always had an idea for lifting vocals from tracks, but i've never really researched it or tried it because i've never actually had the need to do it for my own music, but conceptually it seems like it would be possible. basically, you'd be using phase frequency cancellation. it would only work if you had an instrumental version of the track, of course, but in theory, wouldn't you be able to use the instrumental track, combine it with the full track at a slight phase to cancel out all the like frequencies and be left with just the clean vocal track?

this is an old technique. you have to be careful with sounds that are panned though.

  loganfive said:

  catsonearth said:

i've always had an idea for lifting vocals from tracks, but i've never really researched it or tried it because i've never actually had the need to do it for my own music, but conceptually it seems like it would be possible. basically, you'd be using phase frequency cancellation. it would only work if you had an instrumental version of the track, of course, but in theory, wouldn't you be able to use the instrumental track, combine it with the full track at a slight phase to cancel out all the like frequencies and be left with just the clean vocal track?

 

 

this is an old technique. you have to be careful with sounds that are panned though.

but you could render the track as mono and then back to stereo to get around that?

  catsonearth said:

  loganfive said:

  catsonearth said:

i've always had an idea for lifting vocals from tracks, but i've never really researched it or tried it because i've never actually had the need to do it for my own music, but conceptually it seems like it would be possible. basically, you'd be using phase frequency cancellation. it would only work if you had an instrumental version of the track, of course, but in theory, wouldn't you be able to use the instrumental track, combine it with the full track at a slight phase to cancel out all the like frequencies and be left with just the clean vocal track?

 

 

this is an old technique. you have to be careful with sounds that are panned though.

 

but you could render the track as mono and then back to stereo to get around that?

in which case you'd just phase cancel everything.

  catsonearth said:

i've always had an idea for lifting vocals from tracks, but i've never really researched it or tried it because i've never actually had the need to do it for my own music, but conceptually it seems like it would be possible. basically, you'd be using phase frequency cancellation. it would only work if you had an instrumental version of the track, of course, but in theory, wouldn't you be able to use the instrumental track, combine it with the full track at a slight phase to cancel out all the like frequencies and be left with just the clean vocal track?

i believe so, it would be like using the patch tool or the pattern stamp in photoshop