Deezer Releases Spleeter, An Audio Separation App Library

https://waxy.org/2019/11/fast-and-free-music-separation-with-deezers-machine-learning-library/

  Quote

Cleanly isolating vocals from drums, bass, piano, and other musical accompaniment is the dream of every mashup artist, karaoke fan, and producer. Commercial solutions exist, but can be expensive and unreliable. Techniques like phase cancellation have very mixed results.

The engineering team behind streaming music service Deezer just open-sourced Spleeter, their audio separation library built on Python and TensorFlow that uses machine learning to quickly and freely separate music into stems.

I don;t know, I let this run in 5stem mode on a 6 minute track and all it gave me was two stems - an "other" stem that sounded like what happens when you monitor what's being taken out by a noise reduction plugin, and a piano stem hat was completely silent because I guess they were being very literal about "piano" and combo organ doesn't count.  Those two stems took about 20 minutes to calculate and I let it run all night but it still hadn't calculated any more 8 hours later (it was still processing).  I'll try again with a different track but I'm not expecting much at this point.

EDIT: tried it using the default 2stem settings and the included example audio and this time it ran VERY quickly but it only produced one of the two stems (vocals) and it sounded TERRIBLE, like it had been encoded as a 64kbps mp3 multiple times and then someone had run a first gen FFT nosie reduction plugin on it with really aggressive settings to try to remove the mp3 artifacts.

So yeah, not living up to the hype here. But living up perfectly to all of my experiences with machine learning thus far.  I guess the Deepdream stuff was kind of fun for a while until you noticed that everything it did looked more or less the same.

  On 11/7/2019 at 4:10 PM, TubularCorporation said:

I don;t know, I let this run in 5stem mode on a 6 minute track and all it gave me was two stems - an "other" stem that sounded like what happens when you monitor what's being taken out by a noise reduction plugin, and a piano stem hat was completely silent because I guess they were being very literal about "piano" and combo organ doesn't count.  Those two stems took about 20 minutes to calculate and I let it run all night but it still hadn't calculated any more 8 hours later (it was still processing).  I'll try again with a different track but I'm not expecting much at this point.

 

EDIT: tried it using the default 2stem settings and the included example audio and this time it ran VERY quickly but it only produced one of the two stems (vocals) and it sounded TERRIBLE, like it had been encoded as a 64kbps mp3 multiple times and then someone had run a first gen FFT nosie reduction plugin on it with really aggressive settings to try to remove the mp3 artifacts.

 

So yeah, not living up to the hype here. But living up perfectly to all of my experiences with machine learning thus far.  I guess the Deepdream stuff was kind of fun for a while until you noticed that everything it did looked more or less the same.

 

Expand  

What about the samples they provide? Are you saying those were touched up perhaps?

I don't know, I just know that whether I supplied my own audio or used their encluded example, it didn't work right (only produced some of the stems it was supposed to) and the stems it DID make sounded pretty bad, and that's kind of in line with all of the other machine learning based audio tools I've tried - they usually work very well for a small subset of stuff you throw at them but for mos things they aren't that impressive.  But maybe there's something wrong with my install or something.

I mean, it could definitely have its uses if I could get it to actually save all of the stems, but what I was hearing makes the linked article sound pretty hyperbolic. The examples in that article (I didn't actually get to listen to them before I tried it) don't sound that great either, TBH, the artifacts on those isolated vocals are pretty intense; the backing tracks to fare a lot better so hopefully it will be useful for isolating beats.  It's a cool trick but as it is now you'd really have to want everything sound like it was running through the "reverse" setting on an old Microverb and then used an FFT based noise reduction plugin to try to take the reverb back out to actually use it.  Which could work for somethings but I don't know, I was underwhelmed. I also notice that all of their samples are stuff that has a lot of separation to begin with and is mixed in a pretty conventional way (everything that's older sounds like it was done from modern remasters, too, not the original mixes that actually had personality).

For reference, I tried it on a live-in-studio recording of War from the early 70s that has a great drum sound/performance I would love to sample, and this is the only audio I got (in 5 stem mode it gave me this and a "piano" stem that was completely silent and no other files):

https://www.dropbox.com/s/2en7paosbg0dlhm/other.mp3?dl=1