gptsovit tensorRT推理加速笔记
2026/7/5 16:07:54 网站建设 项目流程

文章目录

    • 一、环境准备
      • 1)模型下载
      • 2)pth模型导出为onnx模型
      • 3)安装trt把onnx文件转为trt

一、环境准备

1)模型下载

模型放在/root/autodl-tmp,I/O读取会快,项目放在根目录,需要创建软连接,还要创建onnx导出目录

cd/root/3rd/GPT-SoVITS_minimal_inferencemkdir-p/root/autodl-tmp/GPT-SoVITS_minimal_inference/pretrained_modelsmkdir-p/root/autodl-tmp/GPT-SoVITS_minimal_inference/onnx_exportrm-rfpretrained_models onnx_exportln-s/root/autodl-tmp/GPT-SoVITS_minimal_inference/pretrained_models pretrained_modelsln-s/root/autodl-tmp/GPT-SoVITS_minimal_inference/onnx_export onnx_export

然后模型都下载/放到这里:
/root/autodl-tmp/GPT-SoVITS_minimal_inference/pretrained_models

##开始下载模型cd/root/3rd/GPT-SoVITS_minimal_inferenceHF_ENDPOINT=https://hf-mirror.com hf download lj1995/GPT-SoVITS\--include"chinese-hubert-base/*"\--include"chinese-roberta-wwm-ext-large/*"\--include"s1v3.ckpt"\--include"v2Pro/s2Gv2ProPlus.pth"\--include"sv/pretrained_eres2netv2w24s4ep4.ckpt"\--local-dir pretrained_models#下载完可以检查:ls-lhpretrained_modelsls-lhpretrained_models/v2Prols-lhpretrained_models/sv

下载速度慢,换成modelscope国内源

cd/root/3rd/GPT-SoVITS_minimal_inference python -<<'PY' from modelscope import snapshot_download snapshot_download( 'dienstag/chinese-roberta-wwm-ext-large', local_dir='pretrained_models/chinese-roberta-wwm-ext-large' ) snapshot_download( 'innnky/chinese-hubert-base-tencent', local_dir='pretrained_models/chinese-hubert-base' ) PY

2)pth模型导出为onnx模型

由于基模不能导出为onnx模型,补充下载模型文件

cd/root/3rd/GPT-SoVITS_minimal_inference#pretrained_models/GPT_weights_v2ProPlus/*.ckpt#pretrained_models/SoVITS_weights_v2ProPlus/*.pthHF_HUB_DISABLE_XET=1python -<<'PY' from huggingface_hub import hf_hub_download import os import shutil repo_id = "lj1995/GPT-SoVITS" downloads = [ ("s1v3.ckpt", "pretrained_models/GPT_weights_v2ProPlus/s1v3.ckpt"), ("v2Pro/s2Gv2ProPlus.pth", "pretrained_models/SoVITS_weights_v2ProPlus/s2Gv2ProPlus.pth"), ("sv/pretrained_eres2netv2w24s4ep4.ckpt", "pretrained_models/sv/pretrained_eres2netv2w24s4ep4.ckpt"), ] for filename, target in downloads: os.makedirs(os.path.dirname(target), exist_ok=True) src = hf_hub_download(repo_id=repo_id, filename=filename) shutil.copy2(src, target) print(f"saved: {target}") PY

导出onnx模型

cd/root/3rd/GPT-SoVITS_minimal_inference pipinstall-rrequirements.txt python export_onnx.py\--gpt_path"pretrained_models/GPT_weights_v2ProPlus/s1v3.ckpt"\--sovits_path"pretrained_models/SoVITS_weights_v2ProPlus/s2Gv2ProPlus.pth"\--cnhubert_base_path"pretrained_models/chinese-hubert-base"\--bert_path"pretrained_models/chinese-roberta-wwm-ext-large"\--sv_path"pretrained_models/sv/pretrained_eres2netv2w24s4ep4.ckpt"\--output_dir"onnx_export/v2proplus_base"\--max_len1000
  • 报错
ImportError: libcudart.so.13 由于 onnxruntime-gpu1.27.0 pip uninstall-yonnxruntime-gpu onnxruntime pipinstall--no-cache-dir onnxruntime-gpu==1.22.0
  • 开始导出onnx模型
(base)root@autodl-container-943b48886a-a5c25e1d:~/3rd/GPT-SoVITS_minimal_inference# python export_onnx.py \--gpt_path"pretrained_models/GPT_weights_v2ProPlus/s1v3.ckpt"\--sovits_path"pretrained_models/SoVITS_weights_v2ProPlus/s2Gv2ProPlus.pth"\--cnhubert_base_path"pretrained_models/chinese-hubert-base"\--bert_path"pretrained_models/chinese-roberta-wwm-ext-large"\--sv_path"pretrained_models/sv/pretrained_eres2netv2w24s4ep4.ckpt"\--output_dir"onnx_export/v2proplus_base"\--max_len1000Loading models... Exporting to onnx_export/v2proplus_base... Exporting SSL... Exporting BERT... Exporting VQEncoder... Exporting GPT Encoder... Exporting GPT Step... Exporting SoVITS... Exporting Spectrogram... min value is tensor(-3.8879)max value is tensor(4.3243)Exporting SV Embedding... Export complete!Config saved to onnx_export/v2proplus_base/config.json

转 FP16:

python onnx_to_fp16.py\--input_dir"onnx_export/v2proplus_base"\--output_dir"onnx_export/v2proplus_base_fp16"....................Saved: onnx_export/v2proplus_base_fp16/gpt_encoder.onnx Processing: gpt_step.onnx|Strategy: FP16(Mixed)[FP16]Converting to FP16...[Attribute Fix]Fixed49attributes(Random/Cast mismatch). Simplifying... Saved: onnx_export/v2proplus_base_fp16/gpt_step.onnx Processing: sovits.onnx|Strategy: FP16(Mixed)[FP16]Converting to FP16... /root/miniconda3/lib/python3.12/site-packages/onnxconverter_common/float16.py:63: UserWarning: the float32 number-10000.0will be truncated to-10000.0warnings.warn([Attribute Fix]Fixed45attributes(Random/Cast mismatch). Simplifying... Saved: onnx_export/v2proplus_base_fp16/sovits.onnx Processing: spectrogram.onnx|Strategy: FP32(Keep)[FP16]Skipping FP16 conversion(Sensitivity/Low-Cost). Simplifying... Saved: onnx_export/v2proplus_base_fp16/spectrogram.onnx Processing: sv_embedding.onnx|Strategy: FP32(Keep)[FP16]Skipping FP16 conversion(Sensitivity/Low-Cost). Simplifying... Saved: onnx_export/v2proplus_base_fp16/sv_embedding.onnx Optimization complete: onnx_export/v2proplus_base_fp16

3)安装trt把onnx文件转为trt

  • 下载trt 10.16版本
mkdir-p/root/3rd/trtcd/root/3rd/trtwget-Onv-tensorrt-local-repo-ubuntu2204-10.16.1-cuda-13.2_1.0-1_amd64.deb\"https://developer.download.nvidia.com/compute/tensorrt/10.16.1/local_installers/nv-tensorrt-local-repo-ubuntu2204-10.16.1-cuda-13.2_1.0-1_amd64.deb"下载完安装: dpkg-inv-tensorrt-local-repo-ubuntu2204-10.16.1-cuda-13.2_1.0-1_amd64.debcp/var/nv-tensorrt-local-repo-ubuntu2204-10.16.1-cuda-13.2/nv-tensorrt-local-*-keyring.gpg /usr/share/keyrings/apt-getupdateapt-getinstall-ytensorrt libnvinfer-dev libnvinfer-plugin-dev libnvonnxparsers-dev libnvinfer-bin 验证:whichtrtexec trtexec--version
  • 下载trt后转成引擎文件
再回到 ONNX 项目转 TRT:cd/root/3rd/GPT-SoVITS_minimal_inference python onnx2trt.py\--input_dir"onnx_export/v2proplus_base_fp16"\--output_dir"onnx_export/v2proplus_base_trt_fp16"\--precisionfp16\--shape_profilefitted

需要专业的网站建设服务?

联系我们获取免费的网站建设咨询和方案报价,让我们帮助您实现业务目标

立即咨询