Need help with blow?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

joansj
129 Stars 21 Forks Apache License 2.0 5 Commits 0 Opened issues

Description

Code to train and run Blow

Services available

!
?

Need anything else?

Contributors list

No Data

Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion

Info

Abstract

End-to-end models for raw audio generation are a challenge, specially if they have to work with non-parallel data, which is a desirable setup in many situations. Voice conversion, in which a model has to impersonate a speaker in a recording, is one of those situations. In this paper, we propose Blow, a single-scale normalizing flow using hypernetwork conditioning to perform many-to-many voice conversion between raw audio. Blow is trained end-to-end, with non-parallel data, on a frame-by-frame basis using a single speaker identifier. We show that Blow compares favorably to existing flow-based architectures and other competitive baselines, obtaining equal or better performance in both objective and subjective evaluations. We further assess the impact of its main components with an ablation study, and quantify a number of properties such as the necessary amount of training data or the preference for source or target speakers.

Reference

J. Serrà, S. Pascual, & C. Segura (2019). Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion. In Advances in Neural Information Processing Systems (NeurIPS). In press.

@article{Serra19ARXIV,
author = {Serr{\`{a}}, J. and Pascual, S. and Segura, C.},
journal = {ArXiv},
title = {{Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion}},
volume = {1906.00794},
year = {2019}
}

Links

Paper: https://arxiv.org/abs/1906.00794 (latest version)

Audio examples: https://blowconversions.github.io

Installation

Suggested steps are:

  1. Clone repository.
  2. Create a conda environment (you can use the
    environment.yml
    file).
  3. The following folder structure will be produced by the repo. From the git folder:
    • src/
      : Where all scripts lie.
    • dat/
      : Place to put all preprocessed files (in subfolders).
    • res/
      : Place to save results.

Running the code

All the following instructions assume you run them from the

src
folder. Also, check the arguments/code for the scripts below. You may want to run with a different configuration.

Preprocessing

To preprocess the audio files:

python preprocess.py --path_in=/path/to/wav/root/folder/ --extension=.wav --path_out=../dat/pt/vctk
Our code expects audio filenames to be in the form
__whatever.extension
, where elements inside
<>
do not contain the character
_
and IDs need not to be consecutive (example:
s001_u045_xxx.wav
). Therefore, if your data is not in this format, you should run or adapt the script
misc/rename_dataset.py
.

Training

To train Blow:

python train.py --path_data=../dat/pt/vctk/ --path_out=../res/vctk/blow/ --model=blow

Synthesis

To transform/synthesize audio with a given learnt model:

python synthesize.py --path_model=../res/vctk/blow/ --path_out=../res/vctk/blow/audio/ --convert

Other

To execute the classification script: ``` python classify.py --mode=train --pathin=../dat/wav/vctk/train/ --fncla=../res/vctk/classif/trainedmodel.pt --fnres=../res/vctk/classif/res_train.pt

python classify.py --mode=test --pathin=../res/vctk/blow/audio/ --fncla=../res/vctk/classif/trainedmodel.pt --fnres=../res/vctk/classif/res_test.pt ```

To listen to some conversions (using sox's

play
command):
python misc/listening_test.py --path_refs_train=../dat/wav/vctk/train/ --path_refs_test=../dat/wav/vctk/test/ --paths_convs=../res/blow/audio/,../res/test1/audio/ --player=play

Notes

  • If using this code, parts of it, or developments from it, please cite the above reference.
  • We do not provide any support or assistance for the supplied code nor we offer any other compilation/variant of it.
  • We assume no responsibility regarding the provided code.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.