Tom Baran - Autotalent

Tom Baran, Ph.D.

Co-founder and CEO, Lumii

Research Affiliate, DSP Group, MIT RLE

Autotalent v0.2

"Pop music in a can!"

What is Autotalent?
Autotalent began as the result of a week of recreational signal processing in May 2009. It's a real-time pitch correction plugin. You specify the notes that a singer is allowed to hit, and Autotalent makes sure that they do. You can also use Autotalent for more exotic effects, like the Cher / T-Pain effect, making your voice sound like a chiptune, adding artificial vibrato, or messing with your formants. Autotalent can also be used as a harmonizer that knows how to sing in the scale with you. Or, you can use Autotalent to change the scale of a melody between major and minor or to change the musical mode.

Can Autotalent be used live?
It works in real-time on my netbook, so probably. The minimum theoretical latency of Autotalent at a sampling rate of 48 kHz is 43ms.

How do I use it?
Autotalent was designed from the ground up to process musical melodies, whether sung by voice or played on some kind of instrument. As long as you give it a monophonic source (i.e. a melody in a single, 1-channel track), you can usually expect pretty good results. So this means that if you're in the business of processing vocal harmonies, you should apply it to each monophonic vocal part separately. If in the spirit of experimentation you're daring enough to try tuning or pitch shifting an entire chord using one instance of the plugin (again not what it's meant for), you'll hear gritty nastiness.

For detailed instructions, have a look at the reference card [PDF], which contains descriptions of the control inputs and outputs, as well as a table of musical scales.

How does it work?
It consists mainly of a pitch detector and a pitch shifter. The pitch detector figures out what pitch the person sang, and based upon this and the values of the various controls, the pitch shifter is instructed to shift the pitch up or down by some appropriate amount, resulting in an output signal that's in tune. Both the pitch detector and pitch shifter are designed to operate on monophonic signals. The pitch detector finds the pitch period via an autocorrelation method, and the pitch shifter uses a time-domain overlap-add technique that's synchronous with the pitch period of the input, which tends to have significantly fewer artifacts than, e.g. phase-vocoder based techniques for single-pitch sources.

The reference card [PDF] also contains a more detailed description of the signal flow throughout the algorithm.

Will Autotalent work on my computer?
Autotalent is an open-source LADSPA plugin and should in theory work on Linux, Mac or Windows if you compile it for those platforms, although Autotalent v0.2 has only been tested on 64-bit Linux. (Autotalent v0.1 had been tested on 32-bit Linux.)

Can I hear it?
Yes! Here's a demo highlighting the tuning and pitch shifting capabilities of Autotalent v0.2. First you'll hear a meager attempt at singing, followed by the same hook processed using several instances of Autotalent in parallel. [mp3]

Things improved in v0.2 from v0.1 (3/20/10):
- More accurate pitch detection.
- Cleaner pitch shifting using polynomial interpolation.
- Formant correction and formant warping added.
- Improved smoothing between note transitions.
- Addition of an extensive LFO / vibrato section.
- Harmonizing now happens in the specified scale, as opposed to happening via a shift up or down by a fixed number of semitones.
- The scale of a melody can be changed between major and minor or to other musical modes via "Output scale rotate".
- The plugin host is now aware of the plugin latency.
- The plugin now has a unique LADSPA ID.
- Detected pitch and confidence have been brought out as control outputs.
- "Pull to fixed pitch" has been added, which allows the user to specify the output pitch of the singer. An enterprising programmer working on an open-source DAW could use this in conjunction with the pitch estimate output to allow users to analyze and modify pitch tracks in their DAW. :)

License update (5/23/10):
The source tar has been updated to include a copy of the Pure Data (BSD-like) license under which the included FFT routines fall. All of the code but the FFT routines is licensed under GPL2.

Where can I get it?
Version 0.2, 64-bit Linux (and reference card): [tar.gz]
Version 0.2, 32-bit Linux (and reference card): [tar.gz]
Version 0.2, source: [tar.gz]
Version 0.2 reference card: [PDF]

Fedora users:
You can install Autotalent from the Fedora Updates repository using yum!
At a shell prompt type
su -c 'yum install ladspa-autotalent-plugins'.
The documentation will be installed to /usr/share/doc/ladspa-autotalent-plugins-0.2/.

Old versions
Version 0.1, 32-bit Linux: [tar.gz]
Version 0.1, source: [tar.gz]

Last updated on December 5, 2015.
This site was made using GNU Emacs.
If you have comments about the site, contact Tom.