本文主要是介绍ChatGLM3-6B独立部署提供HTTP服务failed to open nvrtc-builtins64_121.dll，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
背景
我在本地windoes部署ChatGLM3-bB，且希望部署后能提供HTTP server的能力。
模型部署且启动是成功了，但是在访问生成接口/v1/chat/completions时报错failed to open nvrtc-builtins64_121.dll。
问题详细描述
找不到nvrtc-builtins64_121.dll
RuntimeError: nvrtc: error: failed to open nvrtc-builtins64_121.dll.
Make sure that nvrtc-builtins64_121.dll is installed correctly.
File "D:\ProgramData\Miniconda3\envs\chatglm\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_implreturn self._call_impl(*args, **kwargs)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "D:\ProgramData\Miniconda3\envs\chatglm\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_implreturn forward_call(*args, **kwargs)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "C:\Users\Administrator\.cache\huggingface\modules\transformers_modules\chatglm3-6b\modeling_chatglm.py", line 408, in forwardquery_layer = apply_rotary_pos_emb(query_layer, rotary_pos_emb)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
RuntimeError: nvrtc: error: failed to open nvrtc-builtins64_121.dll.Make sure that nvrtc-builtins64_121.dll is installed correctly.github上也有人报了这个问题
解决方案
原因：pytorch和cuda版本不匹配导致的
pytorch安装时，对应的CUDA12.1
但我本地之前安装的是CUDA12.4
因此找不到12.1版本的nvrtc-builtins64_121
方案：安装多版本CUDA，再安装一个CUDA12.1
这篇关于ChatGLM3-6B独立部署提供HTTP服务failed to open nvrtc-builtins64_121.dll的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
