Neural Instrument Cloning from Very Few Samples

This Blocks implementation is an adaptation a report written by Nicolas Jonason and Bob L.T. Sturm.

I've implemented it in Blocks to show off some cool features, such as embedding live ML demos. More on that ahead...

What does this machine learning model do?

It combines techniques from neural voice cloning with musical instrument synthesis. This makes it possible to produce neural instrument synthesisers from just seconds of target instrument audio.

Audio Examples

Here are some real 16 second saxophone recordings:

Here is a generated saxophone recordings:

Inputs to the model

The inputs to the model are:

  • pitch
  • loudness
  • base audio file

Try the model live!
0 100
0 100

Using the model for cloning

You can also use this model a different way, to simply clone the audio file and reconstruct it using machine learning. Here, we'll show a demo of that below:

Thanks for reading this! As you may have realized, all of the "models" in this demo are fake. They are just designed to show you what is possible using Blocks 🤗.

For details of the model, read the original report here.

Details for nerds: this report was "launched" on: