KaldiFeat
August 17, 2020 ยท View on GitHub
KaldiFeat is a light-weight Python library for computing Kaldi-style acoustic features based on NumPy. It might be helpful if you want to:
- Test a pre-trained model on new data without writing shell commands and creating a bunch of files.
- Run a pre-trained model in a new environment without installing Kaldi.
Example
The following codes calculate MFCCs with the same settings in kaldi/egs/voxceleb/v2
import librosa
from kaldifeat import compute_mfcc_feats, compute_vad, apply_cmvn_sliding
# Assume we have a wav file called example.wav whose sample rate is 16000 Hz
data, _ = librosa.load('example.wav', 16000)
# We adopt 16 bits data, thus we need to transform dtype from float to int16 for librosa
data = (data * 32768).astype(np.int16)
raw_mfcc = compute_mfcc_feats(data, sample_frequency=16000, frame_length=25, frame_shift=10, low_freq=20, high_freq=-400, num_mel_bins=30, num_ceps=30, snip_edges=False)
log_energy = raw_mfcc[:, 0]
vad = compute_vad(log_energy, energy_threshold=5.5, energy_mean_scale=0.5, frames_context=2, proportion_threshold=0.12)
mfcc = apply_cmvn_sliding(raw_mfcc, window=300, center=True)[vad]
Supported Functions
compute_fbank_feats
Compute (log) Mel filter bank energies (FBanks) in the same way as kaldi/src/featbin/compute_fbank_feats
| Parameters | Description |
|---|---|
| blackman_coeff | Constant coefficient for generalized Blackman window. (float, default = 0.42) |
| dither | Dithering constant (0.0 means no dither). If you turn this off, you should set the --energy-floor option, e.g. to 1.0 or 0.1 (float, default = 1) |
| energy_floor | Floor on energy (absolute, not relative) in FBANK computation. Only makes a difference if --use-energy=true; only necessary if --dither=0.0. Suggested values: 0.1 or 1.0 (float, default = 0) |
| frame_length | Frame length in milliseconds (float, default = 25) |
| frame_shift | Frame shift in milliseconds (float, default = 10) |
| high_freq | High cutoff frequency for mel bins (if <= 0, offset from Nyquist) (float, default = 0) |
| low_freq | Low cutoff frequency for mel bins (float, default = 20) |
| num_mel_bins | Number of triangular mel-frequency bins (int, default = 23) |
| preemphasis_coefficient | Coefficient for use in signal preemphasis (float, default = 0.97) |
| raw_energy | If true, compute energy before preemphasis and windowing (bool, default = true) |
| remove_dc_offset | Subtract mean from waveform on each frame (bool, default = true) |
| round_to_power_of_two | If true, round window size to power of two by zero-padding input to FFT. (bool, default = true) |
| sample_frequency | Waveform data sample frequency (must match the waveform file, if specified there) (float, default = 16000) |
| snip_edges | If true, end effects will be handled by outputting only frames that completely fit in the file, and the number of frames depends on the frame-length. If false, the number of frames depends only on the frame-shift, and we reflect the data at the ends. (bool, default = true) |
| use_energy | Add an extra energy output. (bool, default = false) |
| use_log_fbank | If true, produce log-filterbank, else produce linear. (bool, default = true) |
| use_power | If true, use power, else use magnitude. (bool, default = true) |
| window_type | Type of window ("hamming"|"hanning"|"povey"|"rectangular"|"sine"|"blackmann") (string, default = "povey") |
| dtype | Type of array (np.float32|np.float64) (dtype or string, default=np.float32) |
compute_mfcc_feats
Compute Mel-frequency cepstral coefficients (MFCCs) in the same way as kaldi/src/featbin/compute_mfcc_feats
| Parameters | Description |
|---|---|
| blackman_coeff | Constant coefficient for generalized Blackman window. (float, default = 0.42) |
| cepstral_lifter | Constant that controls scaling of MFCCs (float, default = 22) |
| dither | Dithering constant (0.0 means no dither). If you turn this off, you should set the --energy-floor option, e.g. to 1.0 or 0.1 (float, default = 1) |
| energy_floor | Floor on energy (absolute, not relative) in MFCC computation. Only makes a difference if --use-energy=true; only necessary if --dither=0.0. Suggested values: 0.1 or 1.0 (float, default = 0) |
| frame_length | Frame length in milliseconds (float, default = 25) |
| frame_shift | Frame shift in milliseconds (float, default = 10) |
| high_freq | High cutoff frequency for mel bins (if <= 0, offset from Nyquist) (float, default = 0) |
| low_freq | Low cutoff frequency for mel bins (float, default = 20) |
| num_ceps | Number of cepstra in MFCC computation (including C0) (int, default = 13) |
| num_mel_bins | Number of triangular mel-frequency bins (int, default = 23) |
| preemphasis_coefficient | Coefficient for use in signal preemphasis (float, default = 0.97) |
| raw_energy | If true, compute energy before preemphasis and windowing (bool, default = true) |
| remove_dc_offset | Subtract mean from waveform on each frame (bool, default = true) |
| round_to_power_of_two | If true, round window size to power of two by zero-padding input to FFT. (bool, default = true) |
| sample_frequency | Waveform data sample frequency (must match the waveform file, if specified there) (float, default = 16000) |
| snip_edges | If true, end effects will be handled by outputting only frames that completely fit in the file, and the number of frames depends on the frame-length. If false, the number of frames depends only on the frame-shift, and we reflect the data at the ends. (bool, default = true) |
| use_energy | Use energy (not C0) in MFCC computation (bool, default = true) |
| window_type | Type of window ("hamming"|"hanning"|"povey"|"rectangular"|"sine"|"blackmann") (string, default = "povey") |
| dtype | Type of array (np.float32|np.float64) (dtype or string, default=np.float32) |
apply_cmvn_sliding
Apply sliding-window cepstral mean (and optionally variance) normalization in the same way as kaldi/src/featbin/apply_cmvn_sliding
| Parameters | Description |
|---|---|
| center | If true, use a window centered on the current frame (to the extent possible, modulo end effects). If false, window is to the left. (bool, default = false) |
| window | Window in frames for running average CMN computation (int, default = 600) |
| min_window | Minimum CMN window used at start of decoding (adds latency only at start). Only applicable if center == false, ignored if center==true (int, default = 100) |
| norm_vars | If true, normalize variance to one. (bool, default = false) |
compute_vad
Apply energy-based voice activity detection in the same way as kaldi/src/ivectorbin/compute_vad
| Parameters | Description |
|---|---|
| energy_mean_scale | If this is set to s, to get the actual threshold we let m be the mean log-energy of the file, and use s*m + vad-energy-threshold (float, default = 0.5) |
| energy_threshold | Constant term in energy threshold for VAD (also see energy_mean_scale) (float, default = 5) |
| frames_context | Number of frames of context on each side of central frame, in window for which energy is monitored (int, default = 0) |
| proportion_threshold | Parameter controlling the proportion of frames within the window that need to have more energy than the threshold (float, default = 0.6) |