GPU-Accelerated Speech to Text with Kaldi: A Tutorial on Getting Started

Sign up for the latest Speech AI news from NVIDIA.

Recently, NVIDIA achieved GPU-accelerated speech-to-text inference with exciting performance results. That post described the general process of the Kaldi ASR pipeline and indicated which of its elements the team accelerated, that is, implementing the decoder on the GPU and taking advantage of Tensor Cores in the acoustic model.

Now with the latest Kaldi container on NGC, the team has accelerated an additional element, feature extraction, resulting in even faster transcription of audio recordings for Pascal and newer GPUs.

This post is about how to get up and running. We walk you through reproducing our most recent performance benchmark with the GPU-accelerated LibriSpeech and ASpIRE automatic speech recognition (ASR) models, which transcribe audio recordings of speech into text. We also show you how to perform ASR on your own audio recordings using the accelerated model.

Quick background

Kaldi is an open-source software framework for speech processing, the first stage in the conversational AI pipeline, that originated in 2009 at Johns Hopkins University with the intent to develop techniques to reduce both the cost and time required to build speech recognition systems. Kaldi has since grown to become the default speech recognition toolkit in the community, helping enable speech services used by millions of people each day.

The traditional speech-to-text workflow (Figure 1) takes place in the following primary phases:

• Feature extraction (converts a raw audio signal into spectral features suitable for machine learning tasks)
• Acoustic modeling
• Language modeling

Today’s acoustic models in Kaldi replace Gaussian Mixture Models (GMMs) and Hidden Markov Models (HMMs) with both recurrent and convolutional neural networks to predict states efficiently, resulting in state-of-the-art performance.

Walk-through: Three Use Cases

We’re going to cover three use cases to help you start using GPU-accelerated Kaldi in a Linux environment with at least one NVIDIA GPU installed. First, as a baseline, we show you how to reproduce our results using the LibriSpeech model. Next, we demonstrate how to swap out the LibriSpeech model for another model, ASpIRE. Lastly, we guide you through the steps to run your own speech audio through the LibriSpeech model to produce a text transcript.

Setup

For each use case, you must have an NGC account. It’s free and only takes a few minutes to set-up.

Accelerated Kaldi is hosted on an NGC as a container, so the first step is to pull it.

Launch a terminal or shell, and at the command line, enter the following:

The output should resemble the following, and you should see your GPUs listed. In DGX Station, we see four V100 GPUs.

Next, grab the NGC container that contains GPU-accelerated Kaldi:

<xx.xx>-py3 varies with the latest release, but use 19.09-py3 for this post.

After a few moments, the pull operation completes, and you’re ready to try the use cases for yourself. Let’s start by having you reproduce our results.

Reproducing our results with the accelerated LibriSpeech model

It’s time to run the Kaldi container in nvidia-docker to reproduce our results using the accelerated LibriSpeech model.

At this point, you’re presented with a command prompt inside the container. That is, any commands you issue are handled by the container, not the shell from which you launched the container. It should look like the following when you run the container:

The first step is to run the command to download example audio recordings, pre-process them by converting them to WAV files, and download the trained LibriSpeech model. We’ve made a shell script for that. Run the commands:

Depending on the speed of your internet connection, this step should take up to 15 minutes.

In this benchmark, we’re using the LibriSpeech model, trained on 1K hours of recordings of people reading English. When the download and setup are complete, your next step executes a script to run the speech-to-text pipeline on the example audio recordings, accelerated by the GPU. At the command line, run the following:

Understanding the Output

The first portion of text that pops out shows several of the parameters used in the benchmark. For example, you see the flag to use the GPU at the beginning. The batch size for GPU processing and the beam width for the Viterbi search are other notable parameters.

Each Group is a separate iteration that we use for timing calculations. At the end of the output, we calculate RealTimeX by dividing the number of seconds of Total Audio by the Aggregate Total Time.

By the way, you can see GPU 0 being exercised in this output from running the nvidia-smi command again in a different shell while the benchmark was running:

On the V100 16GB GPU in our DGX Station, this benchmark took nearly 75 seconds to complete. The throughput compared to real-time is about 4,372x. Not too shabby!

Benchmarking the Aspire ASR model

With our upcoming Kaldi container 19-10, you can also run the benchmark for the accelerated Automated Speech Recognition in Reverberant Environments (ASpIRE) model. ASpIRE was developed from recordings with far-field microphones of people talking on the phone in noisy, echoey rooms. If you think about how your own speech changes between reading a book out loud and conversing with someone interactively, you’ll get the gist of the difference between ASpIRE and LibriSpeech.

The procedure is the same as for LibriSpeech, but run from the ASpIRE directory. Here are the steps:

The output is in the format as with LibriSpeech benchmark, and the performance is about 2Kx real time.

Transcribing your own data using the accelerated LibriSpeech model

Applying Kaldi’s ASR to your own audio is straightforward. Here’s an example using the LibriSpeech model. First, keep in mind that the LibriSpeech model was generated from a corpus of clean, echo-free, high-SNR recordings. Make sure that your audio files were recorded with a headset or with a mic close to the speaking person’s mouth. Finally, ensure that the files are in the format required by the LibriSpeech model:

• .WAV format
• 16 kHz sampling rate (8 kHz for ASpIRE)
• Mono
• 32-bit float

You can use Audacity, a free audio-processing application for Windows, Mac, and Linux, or a command-line tool such as flac2wav to convert your audio, if necessary. Here’s how my correctly formatted audio file named DT2.wav looked in Audacity. Check the format info at the bottom left.

Second, copy or move your audio files into a single folder, such as /home/user/audio. You could’ve done this step first as the order doesn’t matter in this case.

Next, create a symbolic link from inside the Kaldi container to your audio directory, while also starting up the container. In the shell command, replace /home/user/audio with your own path to your directory of .wav files.

In the container, next change directories to the /librispeech directory:

Almost there! Run the data preparation step:

This step takes around 5 to 15 minutes to finish, as there are several large files to pull down. When it finishes, point the ./run_benchmark.sh script to your audio folder:

If you have only a few audio files to transcribe, the speedup won’t reach the same scale as running thousands of files. However, you can simulate having thousands of files by changing the number of iterations to run with your few files. Depending on your audio, you may want to adjust the batch size down from 300 to avoid an out-of-memory error.

In this specific case, with these audio files, the resulting throughput was 3,684.

For the best word error rate and speedup, it’s important to consider which speech model, LibriSpeech or ASpIRE, is appropriate for your audio. LibriSpeech was trained on low-noise recordings of reading aloud, while ASpIRE was trained on slightly echoey recordings while people conversed on the telephone.

When the script is complete, the resulting transcripts are in the /tmp/results/LibriSpeech directory, starting with a directory named 0. Each time you run the benchmark, this directory name increments by one. For example, here’sthe result of my reading aloud from a technical publication, stored in /root/audio/DT1.wav:

The file trans contains multiple transcriptions if more files are in the audio directory.

What’s next

Now that you’ve tried the GPU-accelerated Kaldi speech recognition toolkit for yourself, you may want to learn more about how NVIDIA drastically increased its throughput. For more information about thought process and considerations behind the accelerated implementation, see the Kaldi GPU Acceleration GTC session.

What’s the NVIDIA team working on next? One item is optimizing even more stages of Kaldi’s processing flow, now that the feature extractor is complete. Another task is testing Kaldi on smaller GPUs, such as the NVIDIA Jetson Xavier and NVIDIA Jetson Nano development boards. Stay tuned!