ADNAC: Audio Denoising using Neural Audio Codecs

Project Overview

ADNAC is a research codebase for audio denoising using neural audio codecs and deep learning models. It provides scripts for dataset preparation, model training (including DAC and U-Net baselines), and evaluation using standard audio quality metrics.

---

Directory Structure

code/
        create_train_dataset.py         # Main script for dataset creation (splitting, degradation, metadata)
        preprocess_dataset.py           # Converts .wav pairs to fast-loading .pt tensors
        torch_dataset_processed.py      # PyTorch Dataset for preprocessed .pt files
        train.py                        # Main training script (DAC model)
        model_unet.py                   # U-Net model definition
        ...                             # Other helpers and legacy scripts

        eval.py                         # Evaluation script: runs U-Net and DAC, computes metrics, saves plots
raw_datasets/
        ...                             # Place raw datasets here (see below)
final_dataset_output/
        ...                             # Processed datasets and outputs
requirements.txt
install.sh                         # Automated setup and dataset download script

---

Setup Instructions

1. Clone the Project

git clone <your-project-repository-url>
cd <your-project-directory>

2. Create and Activate a Python Virtual Environment

python3 -m venv .venv
source .venv/bin/activate

3. Install PyTorch with CUDA Support

Visit the official PyTorch website and follow the instructions for your CUDA version.

4. Install Project Dependencies

pip install -r requirements.txt

5. Download and Organize Datasets

Run the setup script to automate dataset downloads and extraction:

bash install.sh

• This will create the required folder structure under raw_datasets/ and guide you through downloading IRMAS, M&N, MUSAN, UrbanSound8K, FSD50K, and RIR datasets.
• Manual steps: For some datasets (e.g., UrbanSound8K curated snippets), you may need to manually place files as instructed by the script.

---

Dataset Preparation

1. Create Noisy/Clean Paired Dataset

From the code/ directory, run:

python create_train_dataset.py

• This script splits clean audio into frames, applies degradations (noise, reverb, artifacts), and saves paired .wav files and a metadata JSON in final_dataset_output/denoising_dataset_dac_musical_2s/.

2. Preprocess Dataset for Fast Training

Convert .wav pairs to .pt tensors for efficient loading:

python preprocess_dataset.py \
        --source_dataset_dir ../final_dataset_output/denoising_dataset_dac_musical_2s \
        --source_metadata_file ../final_dataset_output/denoising_dataset_dac_musical_2s/dataset_metadata.json \
        --processed_dataset_dir ../final_dataset_output/denoising_dataset_dac_musical_2s \
        --target_sample_rate 44100

• This creates dataset_metadata_processed.json and .pt files for each pair.

---

Training

Train the DAC-based denoising model:

python train.py \
        --base_dataset_path ../final_dataset_output/denoising_dataset_dac_musical_2s \
        --output_path ./training_outputs \
        --run_name my_experiment

• Adjust --batch_size, --epochs, and other hyperparameters as needed.
• Training logs and checkpoints are saved in training_outputs/<run_name>/.

---

Evaluation

Evaluate the trained models (DAC and U-Net baseline) on the test set:

python evaluation/eval.py \
        --test_dir ../final_dataset_output/denoising_dataset_dac_musical_2s/test \
        --unet_model_path <path_to_trained_unet.pth> \
        --dac_model_path <path_to_finetuned_dac.pt> \
        --output_dir ./evaluation_results

• Computes SI-SDR, PESQ, and STOI metrics for both models.
• Saves results as CSV and spectrogram comparison plots.

---

Code Structure Overview

• Dataset Creation: create_train_dataset.py
  Loads clean audio, applies degradations (noise, reverb, artifacts), and saves noisy/clean pairs with metadata.
• Preprocessing: preprocess_dataset.py
  Converts .wav pairs to .pt tensors for fast PyTorch loading.
• Dataset Loader: torch_dataset_processed.py
  PyTorch Dataset for loading preprocessed pairs.
• Training: train.py
  Trains the DAC model using L1, Mel, and SI-SDR losses.
• Evaluation: evaluation/eval.py
  Loads U-Net and DAC models, runs inference, computes metrics, and saves plots.

---

Tips

• Always activate your virtual environment before running scripts:

  source .venv/bin/activate
  

• For GPU training, verify CUDA is available:

  import torch
  print(torch.cuda.is_available())
  

• Adjust dataset paths in create_train_dataset.py if your folder structure differs.

---

License

MIT License. See LICENSE for details.