Clone a voice in 5 seconds to generate arbitrary speech in real-time

已是最新代码 E:\MockingBird\MockingBird>python demo_toolbox.py Traceback (most recent call last): File "E:\MockingBird\MockingBird\demo_toolbox.py", line 2, in from toolbox import Toolbox File "E:\MockingBird\MockingBird\toolbox_init_.py", line 9, in from utils.f0_utils import compute_f0, f02lf0, compute_mean_std, get_converted_lf0uv File "E:\MockingBird\MockingBird\utils\f0_utils.py", line 3, in import pyworld ModuleNotFoundError: No module named 'pyworld'

问题简述 使用自己的数据集训练合成器模型的时候的时候，在预处理之后训练合成器并将合成器替换成既有model后产生的图并没有收敛。

复现与环境

参照www.bilibili.com/video/BV1dq4y137pH 进行的复现。代码版本为main branch，首先进行数据预处理之后参考视频里的首先进行合成器训练，然后用pretrained-11-7-21 替换掉当前mode 继续进行训练。发现图并没有收敛。 截图

你好 我已經下載了aidatatang_200zh這個數據集，並且把 aidatatang_200zh\corpus\train 底下的檔案都解壓縮完畢 但是當我要開始執行 python synthesizer_preprocess_audio.py D:\google download(我把檔案放在 D:\google download 這個路徑下 ) 卻發生以下狀況: D:\python_demo\Realtime-Voice-Clone-Chinese>python synthesizer_preprocess_audio.py D:\google download\ D:\python_demo\Realtime-Voice-Clone-Chinese\encoder\audio.py:13: UserWarning: Unable to import 'webrtcvad'. This package enables noise removal and is recommended. warn("Unable to import 'webrtcvad'. This package enables noise removal and is recommended.") usage: synthesizer_preprocess_audio.py [-h] [-o OUT_DIR] [-n N_PROCESSES] [-s] [--hparams HPARAMS] [--no_trim] [--no_alignments] [--dataset DATASET] datasets_root synthesizer_preprocess_audio.py: error: unrecognized arguments: download\

請問我可以怎麼解決問題呢? 我有查看之前 issues 的討論並沒有發現有類似問題，以下是我想到可能有問題的地方，還請作者為我解答，謝謝！

1.我只有解壓縮 aidatatang_200zh\corpus\train 底下的檔案，是否其他資料夾下的檔案也要解壓縮? 2.是不是只需要將所有 wav 檔單獨拉出來放在 aidatatang_200zh\corpus\train 底下然後再執行python synthesizer_preprocess_audio.py D:\google download ? 3. 輸入的指令不對 4. wav 檔 與 txt 檔是不是要預先處理，而我沒有進行處理?

用社区分享的模型训练报错 不知道原因 而且不知道咋保存模型 是不是必须要每500步才会自动保存 求各位大佬解惑 感谢！ RuntimeError: The size of tensor a (1024) must match the size of tensor b (3) at non-singleton dimension 3

Variable._execution_engine.run_backward(RuntimeError: CUDA out of memory. Tried to allocate 88.00 MiB (GPU 0; 4.00 GiB totalcapacity; 2.68 GiB already allocated; 0 bytes free; 2.85 GiB reserved in total by PyTorch)

能不能提供一个调batch_size的参数? 我目前用的显卡显存只有4G(GTX1050Ti)，默认参数正常训练时经常爆掉显存....

我在运行 python synthesizer_preprocess_audio.py时遇到如上错误 ，在CSDN上找到解决方法：1.如果python 运行环境不在C盘 查看高级系统设置->高级->性能 设置->高级->虚拟内存->更改 ->取消自动管理所有驱动器的分页文件大小-> 自定义大小 ->初始大小和最大值设为10240 2. 更改DateLoade 中的参数num_worker 改为0 但我现在不清楚具体怎样把参数设为0

Win11 GPU：3060laptop
Python 3.9.13

+----------------+------------+---------------+------------------+ | Steps with r=2 | Batch Size | Learning Rate | Outputs/Step (r) | +----------------+------------+---------------+------------------+ | 101k Steps | 16 | 3e-06 | 2 | +----------------+------------+---------------+------------------+

Could not load symbol cublasGetSmCountTarget from cublas64_11.dll. Error code 127 Traceback (most recent call last): File "G:\AIvioce\MockingBird\synthesizer_train.py", line 37, in train(**vars(args)) File "G:\AIvioce\MockingBird\synthesizer\train.py", line 216, in train optimizer.step() File "C:\Users\Mark\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\optimizer.py", line 109, in wrapper return func(*args, **kwargs) File "C:\Users\Mark\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "C:\Users\Mark\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\adam.py", line 157, in step adam(params_with_grad, File "C:\Users\Mark\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\adam.py", line 213, in adam func(params, File "C:\Users\Mark\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\adam.py", line 255, in _single_tensor_adam assert not step_t.is_cuda, "If capturable=False, state_steps should not be CUDA tensors." AssertionError: If capturable=False, state_steps should not be CUDA tensors.

以下是报错代码： Building wheels for collected packages: ctc-segmentation, pyworld Building wheel for ctc-segmentation (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [12 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.7 creating build\lib.win-amd64-3.7\ctc_segmentation copying ctc_segmentation\ctc_segmentation.py -> build\lib.win-amd64-3.7\ctc_segmentation copying ctc_segmentation\partitioning.py -> build\lib.win-amd64-3.7\ctc_segmentation copying ctc_segmentation_init_.py -> build\lib.win-amd64-3.7\ctc_segmentation running build_ext building 'ctc_segmentation.ctc_segmentation_dyn' extension error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/ [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for ctc-segmentation Running setup.py clean for ctc-segmentation Building wheel for pyworld (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for pyworld (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [13 lines of output] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.7 creating build\lib.win-amd64-3.7\pyworld copying pyworld_init_.py -> build\lib.win-amd64-3.7\pyworld running build_ext skipping 'pyworld\pyworld.cpp' Cython extension (up-to-date) building 'pyworld.pyworld' extension C:\Users\ADMINI~1\AppData\Local\Temp\pip-build-env-_y7fbfzj\overlay\Lib\site-packages\setuptools\dist.py:741: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead % (opt, underscore_opt) error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pyworld Failed to build ctc-segmentation pyworld ERROR: Could not build wheels for pyworld, which is required to install pyproject.toml-based projects

Summary[问题简述（一句话）] 训练模型时这个问题怎么办？？疑似N卡内存不够。 CUDA out of memory. Tried to allocate 122.00 MiB (GPU 0; 4.00 GiB total capacity; 3.15 GiB already allocated; 0 bytes free; 3.45 GiB reserved in total by PyTorch

Env & To Reproduce[复现与环境] python3.9、NVIDIA GeForce GTX 1050Ti（4GB）

Screenshots[截图（如有）]

F:\VideoCentTools\MockingBird-main>python synthesizer_train.py offhen F:\VideoCentTools/SV2TTS/synthesizer Traceback (most recent call last): File "F:\VideoCentTools\MockingBird-main\synthesizer_train.py", line 2, in from synthesizer.train import train File "F:\VideoCentTools\MockingBird-main\synthesizer\train.py", line 5, in from torch.utils.tensorboard import SummaryWriter File "C:\Users\Administrator\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\utils\tensorboard_init_.py", line 4, in LooseVersion = distutils.version.LooseVersion AttributeError: module 'setuptools._distutils' has no attribute 'version'

我把已经下载好的模型，放到了文件D:\声音克隆\MockingBird-main\synthesizer\saved_models下 并且还在D:\声音克隆\MockingBird-main\encoder\saved_models里也放了一个把模型my_run,py改名为pretrained.pt的文件 然后运行web.py文件

(base) C:\Users\13549>python D:\声音克隆\MockingBird-main\web.py Loaded synthesizer models: 0 Traceback (most recent call last): File "D:\声音克隆\MockingBird-main\web.py", line 6, in app = webApp() File "D:\声音克隆\MockingBird-main\web_init_.py", line 33, in webApp encoder.load_model(Path("encoder/saved_models/pretrained.pt")) File "D:\声音克隆\MockingBird-main\encoder\inference.py", line 33, in load_model checkpoint = torch.load(weights_fpath, _device) File "D:\anaconda\lib\site-packages\torch\serialization.py", line 525, in load with _open_file_like(f, 'rb') as opened_file: File "D:\anaconda\lib\site-packages\torch\serialization.py", line 212, in _open_file_like return _open_file(name_or_buffer, mode) File "D:\anaconda\lib\site-packages\torch\serialization.py", line 193, in init super(_open_file, self).init(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: 'encoder\saved_models\pretrained.pt'

请问一下该怎么办呢，即使我把模型文件名修改成pretrained仍然会报同样的错误

今天刚搭建的环境，使用VC拟音时遇到 ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject 此时python=3.9.15, numpy=1.19.3, pyworld=0.3.2 查看报错信息发现和pyworld有关，将pyworld版本降低到0.3.0后不再报错。可能需要在requirements.txt限制pyworld版本？

Summary[问题简述（一句话）] A clear and concise description of what the issue is.

训练到18K，attention图不显示

Env & To Reproduce[复现与环境] 描述你用的环境、代码版本、模型

win10，anaconda虚拟环境python=3.9.7，代码版本是在2022年12月20日下载的，数据集是aidatatang_200zh。

电脑是MacBook Pro 2019

CPU: Intel(R) Core(TM) i9-9880H @2.3GHz

Screenshots[截图（如有）] If applicable, add screenshots to help

Summary[问题简述（一句话）] 在Windows上运行出现问题：module 'umap' has no attribute 'UMAP' 在训练声码器和启动demo时都出现了这个问题

Env & To Reproduce[复现与环境]

Traceback (most recent call last):
  File "D:\MockingBird\encoder_train.py", line 46, in <module>
    train(**vars(args))
  File "D:\MockingBird\encoder\train.py", line 100, in train
    vis.draw_projections(embeds, utterances_per_speaker, step, projection_fpath)
  File "D:\MockingBird\encoder\visualizations.py", line 164, in draw_projections
    reducer = umap.UMAP()
AttributeError: module 'umap' has no attribute 'UMAP'

加载数据集运行工具箱时报错： AttributeError: 'Toolbox' object has no attribute 'selected_source_utterance'

工具箱右上边，输入框下边的Vocode only按钮也是灰色的，只能合成没有声音输出

工具箱左下角 Toolbox outpup 也是无法加载选项

当前版本：@main-b402f9d 时间：2022-12-15

当前 requirements.txt 中所缺失的包：

fastapi==0.88.0
  pydantic==1.10.2
    typing_extensions==4.4.0
  starlette==0.22.0
    anyio==3.6.2
      idna==3.4
      sniffio==1.3.0
    typing_extensions==4.4.0
loguru==0.6.0
  colorama==0.4.6
  win32-setctime==1.1.0
typer==0.7.0
  click==8.0.0
    colorama==0.4.6

注：其中倒数第二行， click 版本应注明保持在 8.0.0 ，新版本会导致 "get_os_args" 的报错。其他包最新版本仍可用