Music Demixing Challenge 2023 - Music Separation - Starter Kit - KUIELab-Mdx-Net Edition

March 13, 2023 ยท View on GitHub

Banner image

Music Demixing Challenge 2023 - Music Separation - Starter Kit - KUIELab-Mdx-Net Edition

Discord

This repository is the Sound Demixing Challenge 2023 - Music Separation Starter kit, KUIELab-MDX-Net Edition ! It contains:

  • How to train a vanilla KUIELab-MDX-Model for the challenge!
  • How to submit a trained KUIELab-MDX-Model to AICrowd's system

This repository does not cover

  • How to make training of KUIELab-MDX-Net more robust on corrupted dataset
  • Training Mixer, which was used in the previous challenge was omitted.

about KUIELab-MDX-Net

A. Quick Start

If you want to submit the pretrained baseline, please click here!

B. How to reproduce this model?

1. Environment

  • Ubuntu 20.04
  • at least two cuda-able GPUs (each >= 2080ti)
  • wandb or tensorboard for logging

Also, you must create .env file by copying .env.sample to set environmental variables.

# Download datasets and link paths to these variables
# https://www.aicrowd.com/challenges/sound-demixing-challenge-2023/problems/music-demixing-track-mdx-23/dataset_files

label_noise_data_dir="/home/git/moisesdb23_labelnoise_v1.0/"
bleeding_data_dir="/home/git/moisesdb23_bleeding_v1.0/"

# You don't have to fill this if you want to use tensorboard only.
wandb_api_key=[YOUR WANDB API KEY] # go wandb.ai/settings and copy your key
  • about wandb_api_key
    • we currently support wandb and tensorboard for logging.
    • for wandb_api_key, visit wandb, go to setting, and then copy your api key
  • about label_noise_data_dir and bleeding_data_dir
    • the absolute path where moisesdb23_*_v* datasets are stored

2. Installation

conda env create -f conda_env_gpu.yaml -n mdx-net-sdx23
conda activate mdx-net-sdx233
pip install -r requirements.txt

3. Training

  • Train four different models: vocals, drums, bass and other

A. for label_noise

  • vocals:
    • python train.py experiment=multigpu_vocals datamodule=moisesdb23_labelnoise_v1 trainer.gpus=2 logger=tensorboard trainer=ddp
  • drums:
    • python train.py experiment=multigpu_drums datamodule=moisesdb23_labelnoise_v1 trainer.gpus=2 logger=tensorboard trainer=ddp
  • bass:
    • python train.py experiment=multigpu_bass datamodule=moisesdb23_labelnoise_v1 trainer.gpus=2 logger=tensorboard trainer=ddp
  • other:
    • python train.py experiment=multigpu_other datamodule=moisesdb23_labelnoise_v1 trainer.gpus=2 logger=tensorboard trainer=ddp

B. for bleeding

  • vocals:
    • python train.py experiment=multigpu_vocals datamodule=moisesdb23_bleeding_v1 trainer.gpus=2 logger=tensorboard trainer=ddp
  • drums:
    • python train.py experiment=multigpu_drums datamodule=moisesdb23_bleeding_v1 trainer.gpus=2 logger=tensorboard trainer=ddp
  • bass:
    • python train.py experiment=multigpu_bass datamodule=moisesdb23_bleeding_v1 trainer.gpus=2 logger=tensorboard trainer=ddp
  • other:
    • python train.py experiment=multigpu_other datamodule=moisesdb23_bleeding_v1 trainer.gpus=2 logger=tensorboard trainer=ddp

Other options

gpus
  • you can use single gpu by setting trainer.gpus=1
  • you can specify the ids of gpus that you want to use trainer.gpus=[1,3]
loggers
  • you can use tensorboard by setting logger=tensorboard
  • you can use wandb by setting logger=wandb
  • you can use both of them by setting logger=['tensorboard,wandb']
batch_sizes
  • you can adjust the size of batch per each node: datamodule.batch_size=2
Model Details

You can change hyperparameters if you want in configs/model/ConvTDFNet_bass.yaml, configs/model/ConvTDFNet_drums.yaml, configs/model/ConvTDFNet_other.yaml, configs/model/ConvTDFNet_vocals.yaml

4. Submission

  • If you think there is no sdr improvement in validation score, then please stop training.

  • During training, configs/callbacks/default.yaml defines model_checkpoint automatically, which stores the checkpoints with the top-5 sdrs.

    • you can set different value for it in configs/callbacks/default.yaml or putting callbacks.model_checkpoint.save_top_k=10 to the shell command.

  • check your checkpoints. for example,

    ls logs/runs/vocals/2023-02-27/16-52-17/checkpoints/
'epoch=00.ckpt'   last.ckpt

A. for label_noise

  1. Please fork this repository and set up. 
    git clone https://gitlab.aicrowd.com/[YourGitLabAccount]/sdx-2023-music-demixing-track-starter-kit
cd sdx-2023-music-demixing-track-starter-kit
git checkout label_noise
ls models/mdx-net-labelnoise/vocals/
  2. it should give 
    config.yaml  epoch=2543.ckpt
  3. you can add your own checkpoints, like this
config.yaml  epoch=2543.ckpt epoch=00.ckpt
  1. open sdx-2023-music-demixing-track-starter-kit/my_submission/mdxnet_labelnoise_music_separation_model.py and link checkpoints.
    def __init__(self):
        self.separator = torch.nn.ModuleDict(
            {
                'vocals': get_model(root_dir='./models/mdx-net-labelnoise/vocals', ckpt_path='epoch=00.ckpt'),
                # 'vocals': get_model(root_dir='./models/mdx-net-labelnoise/vocals', ckpt_path='epoch=2543.ckpt'),
                'bass': get_model(root_dir='./models/mdx-net-labelnoise/bass', ckpt_path='epoch=694.ckpt'),
                'drums': get_model(root_dir='./models/mdx-net-labelnoise/drums', ckpt_path='epoch=408.ckpt'),
                'other': get_model(root_dir='./models/mdx-net-labelnoise/other', ckpt_path='epoch=763.ckpt')
            }
        )
  1. don't forget use git-lfs to upload them to your own repository
git lfs track models/mdx-net-labelnoise/vocals/epoch\=00.ckpt
git add .gitattributes
  1. commit and submit it to aicrowd!
git tag -am "submission-mdx-label-noise" "submission-mdx-label-noise" 
git push

B. for bleeding

  1. Please fork this repository and set up. 
    git clone https://gitlab.aicrowd.com/[YourGitLabAccount]/sdx-2023-music-demixing-track-starter-kit
cd sdx-2023-music-demixing-track-starter-kit
git checkout bleeding
ls models/mdx-net-bleeding/vocals/
  2. it should give 
    config.yaml  epoch=744.ckpt
  3. you can add your own checkpoints, like this
config.yaml  epoch=744.ckpt epoch=00.ckpt
  1. open sdx-2023-music-demixing-track-starter-kit/my_submission/mdxnet_bleeding_music_separation_model.py and link checkpoints.
    def __init__(self):
        self.separator = torch.nn.ModuleDict(
            {
                'vocals': get_model(root_dir='./models/mdx-net-labelnoise/vocals', ckpt_path='epoch=00.ckpt'),
                #'vocals': get_model(root_dir='./models/mdx-net-bleeding/vocals', ckpt_path='epoch=744.ckpt'),
                'bass': get_model(root_dir='./models/mdx-net-bleeding/bass', ckpt_path='epoch=995.ckpt'),
                'drums': get_model(root_dir='./models/mdx-net-bleeding/drums', ckpt_path='epoch=780.ckpt'),
                'other': get_model(root_dir='./models/mdx-net-bleeding/other', ckpt_path='epoch=437.ckpt')
            }
        )
  1. don't forget use git-lfs to upload them to your own repository
git lfs track models/mdx-net-bleeding/vocals/epoch\=00.ckpt
git add .gitattributes
  1. commit and submit it to aicrowd!
git tag -am "submission-mdx-bleeding" "submission-mdx-bleeding" 
git push

ACKNOWLEDGEMENT