启动服务器下载好的本地音频模型报错 #2691

moshenwu · 2024-12-19T10:01:08Z

System Info / 系統信息

cuda version:12.6
Python:3.11.10
vllm=0.6.4.post1

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

docker / docker
pip install / 通过 pip install 安装
installation from source / 从源码安装

Version info / 版本信息

1.1.0

The command used to start Xinference / 用以启动 xinference 的命令

nohup env XINFERENCE_HOME=/home/xinference xinference-local --host 0.0.0.0 --port 9997 > /home/logs/xinference.log 2>&1 &

Reproduction / 复现过程

POST请求 http://localhost:9997/v1/models
入参为 {"model_uid":null,"model_name":"fish-speech","model_type":"audio","replica":1,"n_gpu":"auto","worker_ip":null,"gpu_idx":null,"download_hub":null,"model_path":null}
注意本音频模型已在本地服务器成功注册，返回错误信息如下：
{
"detail": "[address=0.0.0.0:46873, pid=2102541] The checkpoint you are trying to load has model type dual_ar but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date."
}

Expected behavior / 期待表现

期望本地下载的音频模型可以成功启动不报错

The text was updated successfully, but these errors were encountered:

qinxuye · 2024-12-20T02:43:42Z

服务端完整日志贴下。

moshilangzi · 2024-12-23T08:01:04Z

服务端完整日志贴下。

fish-speech-1.4和whisper-large-v3-turbo先下载模型，然后进行注册，然后启动，俩都报错：

fish-speech-1.4报错：
`2024-12-18 17:18:30,016 xinference.core.worker 147420 INFO [request 0bbb2530-bd21-11ef-b6f1-189ba584b5f6] Enter launch_builtin_model, args: <xinference.core.worker.WorkerActor object at 0x7f1f64569190>, kwargs: model_uid=fish-speech-0,model_name=fish-speech,model_size_in_billions=None,model_format=None,quantization=None,model_engine=None,model_type=audio,n_gpu=auto,request_limits=None,peft_model_config=None,gpu_idx=[1],download_hub=None,model_path=/home/models/AI-ModelScope/fish-speech-1.4
2024-12-18 17:18:30,017 xinference.core.worker 147420 INFO You specify to launch the model: fish-speech on GPU index: [1] of the worker: 0.0.0.0:57879, xinference will automatically ignore the n_gpu option.
2024-12-18 17:18:35,271 xinference.core.model 174969 INFO Start requests handler.
2024-12-18 17:18:35,842 transformers.configuration_utils 174969 INFO loading configuration file /home/models/AI-ModelScope/fish-speech-1.4/config.json
2024-12-18 17:18:35,846 xinference.core.worker 147420 ERROR Failed to load model fish-speech-0
Traceback (most recent call last):
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1034, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 736, in getitem
raise KeyError(key)
^^^^^^^^^^^^^^^^^
KeyError: 'dual_ar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/worker.py", line 897, in launch_builtin_model
await model_ref.load()
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send
return self._process_result_message(result)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
raise message.as_instanceof_cause()
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 667, in send
result = await self._run_coro(message.message_id, coro)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
return await coro
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive
return await super().on_receive(message) # type: ignore
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 558, in on_receive
raise ex
File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive
async with self._lock:
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive
with debug_async_timeout('actor_lock_timeout',
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive
result = await result
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/model.py", line 414, in load
self._model.load()
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/model/audio/whisper.py", line 63, in load
model = AutoModelForSpeechSeq2Seq.from_pretrained(
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 526, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1036, in from_pretrained
raise ValueError(
ValueError: [address=0.0.0.0:39903, pid=174969] The checkpoint you are trying to load has model type dual_ar but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
2024-12-18 17:18:35,894 xinference.core.worker 147420 ERROR [request 0bbb2530-bd21-11ef-b6f1-189ba584b5f6] Leave launch_builtin_model, error: [address=0.0.0.0:39903, pid=174969] The checkpoint you are trying to load has model type dual_ar but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date., elapsed time: 5 s`

使用本地下载的模型的路径部署whisper-large-v3-turbo模型，命名whisper-large-v3-turbo-local,启动报错：

2024-12-23 15:54:07,375 xinference.api.restful_api 623349 ERROR [address=0.0.0.0:41357, pid=2131352] The checkpoint you are trying to load has model type dual_ar` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
Traceback (most recent call last):
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1034, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 736, in getitem
raise KeyError(key)
^^^^^^^^^^^^^^^^^
KeyError: 'dual_ar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/api/restful_api.py", line 1002, in launch_model
model_uid = await (await self._get_supervisor_ref()).launch_builtin_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send
return self._process_result_message(result)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
raise message.as_instanceof_cause()
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 667, in send
result = await self._run_coro(message.message_id, coro)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
return await coro
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive
return await super().on_receive(message) # type: ignore
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 558, in on_receive
raise ex
File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive
async with self._lock:
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive
with debug_async_timeout('actor_lock_timeout',
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive
result = await result
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/supervisor.py", line 1041, in launch_builtin_model
await _launch_model()
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/supervisor.py", line 1005, in _launch_model
await _launch_one_model(rep_model_uid)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/supervisor.py", line 984, in _launch_one_model
await worker_ref.launch_builtin_model(
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send
return self._process_result_message(result)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
raise message.as_instanceof_cause()
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 667, in send
result = await self._run_coro(message.message_id, coro)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
return await coro
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive
return await super().on_receive(message) # type: ignore
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 558, in on_receive
raise ex
File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive
async with self._lock:
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive
with debug_async_timeout('actor_lock_timeout',
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive
result = await result
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/utils.py", line 90, in wrapped
ret = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/worker.py", line 897, in launch_builtin_model
await model_ref.load()
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 231, in send
return self._process_result_message(result)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/context.py", line 102, in _process_result_message
raise message.as_instanceof_cause()
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 667, in send
result = await self._run_coro(message.message_id, coro)
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/backends/pool.py", line 370, in _run_coro
return await coro
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xoscar/api.py", line 384, in on_receive
return await super().on_receive(message) # type: ignore
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 558, in on_receive
raise ex
File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive
async with self._lock:
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive
with debug_async_timeout('actor_lock_timeout',
^^^^^^^^^^^^^^^^^
File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.on_receive
result = await result
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/core/model.py", line 414, in load
self._model.load()
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/xinference/model/audio/whisper.py", line 63, in load
model = AutoModelForSpeechSeq2Seq.from_pretrained(
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 526, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/xinference/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1036, in from_pretrained
raise ValueError(
ValueError: [address=0.0.0.0:41357, pid=2131352] The checkpoint you are trying to load has model type dual_ar but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.：`

qinxuye · 2024-12-23T09:01:21Z

pip show transformers 看下版本

moshilangzi · 2024-12-23T10:08:25Z

pip show transformers 看下版本

Name: transformers
Version: 4.46.3
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: [email protected]
License: Apache 2.0 License
Location: /home/anaconda3/envs/xinference/lib/python3.11/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: auto_gptq, autoawq, chattts, compressed-tensors, FlagEmbedding, nemo_text_processing, optimum, peft, sentence-transformers, transformers-stream-generator, vllm

qinxuye · 2024-12-24T11:16:23Z

重新安装 transformers 看下

pip uninstall transformers
pip install transformers --no-cache

moshilangzi · 2024-12-25T01:49:14Z

重新安装 transformers 看下
pip uninstall transformers
pip install transformers --no-cache

环境重装啦，还是不行的，直接选择xinference里面提供的模型，让它自己下载可以正常运行，但是我下载好模型，指定模型路径，然后启动，就报错的。
重装命令： pip install "xinference[all]" -i https://pypi.tuna.tsinghua.edu.cn/simple -U --force-reinstall

qinxuye · 2024-12-25T07:30:31Z

你们模型是在 modelscope 上下载的吗？

moshilangzi · 2024-12-30T08:44:10Z

你们模型是在 modelscope 上下载的吗？

modelscope和huggingface都尝试了，SenseVoiceSmall、FishSpeech-1.5、CosyVoice2-0.5B这些asr和tts的模型，进行自定义部署，需要重新命名，然后指定下载的模型路径进行启动，这个时候报错，但是使用官方提供的配置，然后又直接指定下载的模型路径，是可以正常启动的，证明了模型本身没问题的，是个bug

qinxuye · 2024-12-30T09:15:10Z

你们模型是在 modelscope 上下载的吗？

modelscope和huggingface都尝试了，SenseVoiceSmall、FishSpeech-1.5、CosyVoice2-0.5B这些asr和tts的模型，进行自定义部署，需要重新命名，然后指定下载的模型路径进行启动，这个时候报错，但是使用官方提供的配置，然后又直接指定下载的模型路径，是可以正常启动的，证明了模型本身没问题的，是个bug

所以是 audio 注册自定义会有问题，走内置模型 + model_path 有问题吗？

github-actions · 2025-01-06T19:03:46Z

This issue is stale because it has been open for 7 days with no activity.

XprobeBot added the gpu label Dec 19, 2024

XprobeBot added this to the v1.x milestone Dec 19, 2024

github-actions bot added the stale label Jan 6, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

启动服务器下载好的本地音频模型报错 #2691

启动服务器下载好的本地音频模型报错 #2691

moshenwu commented Dec 19, 2024

qinxuye commented Dec 20, 2024

moshilangzi commented Dec 23, 2024

qinxuye commented Dec 23, 2024

moshilangzi commented Dec 23, 2024

qinxuye commented Dec 24, 2024

moshilangzi commented Dec 25, 2024

qinxuye commented Dec 25, 2024

moshilangzi commented Dec 30, 2024

qinxuye commented Dec 30, 2024

github-actions bot commented Jan 6, 2025

启动服务器下载好的本地音频模型报错 #2691

启动服务器下载好的本地音频模型报错 #2691

Comments

moshenwu commented Dec 19, 2024

System Info / 系統信息

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

Version info / 版本信息

The command used to start Xinference / 用以启动 xinference 的命令

Reproduction / 复现过程

Expected behavior / 期待表现

qinxuye commented Dec 20, 2024

moshilangzi commented Dec 23, 2024

qinxuye commented Dec 23, 2024

moshilangzi commented Dec 23, 2024

qinxuye commented Dec 24, 2024

moshilangzi commented Dec 25, 2024

qinxuye commented Dec 25, 2024

moshilangzi commented Dec 30, 2024

qinxuye commented Dec 30, 2024

github-actions bot commented Jan 6, 2025