使用 modelscope产生的问题和解决方案

使用 modelscope产生的问题和解决方案

flyfish

代码地址https://github.com/QwenLM/Qwen2

这里将原代码做了更改
例如

模型容易下载
from transformers import AutoModelForCausalLM, AutoTokenizer更改为from modelscope import AutoModelForCausalLM, AutoTokenizer

使用小的模型快速测试
"Qwen/Qwen2-7B-Instruct"更改为"qwen/Qwen2-0.5B-Instruct"

完整代码如下

from modelscope import AutoModelForCausalLM, AutoTokenizermodel_name = "qwen/Qwen2-0.5B-Instruct"
device = "cuda" # the device to load the model ontomodel = AutoModelForCausalLM.from_pretrained(model_name,torch_dtype="auto",device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)prompt = "Give me a short introduction to large language model."
messages = [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(messages,tokenize=False,add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)generated_ids = model.generate(**model_inputs,max_new_tokens=512
)
generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

输出
A large language model is an artificial intelligence system that can generate human-like text from natural language input. These models are designed to mimic the way humans process and understand language, and they can generate sentences based on patterns and relationships in the input text. Large language models have been used for a variety of applications, including chatbots, virtual assistants, and automated decision-making systems. They are becoming increasingly important as we move towards more complex and nuanced interactions with machines, and their development has attracted significant attention from researchers and industry leaders alike.

错误1

from modelscope import AutoModelForCausalLM, AutoTokenizer
ImportError: cannot import name 'AutoModelForCausalLM' from 'modelscope'

modelscope版本低，modelscope1.4.3版本解决问题

pip install modelscope==1.4.3 -f https://modelscope.oss-cn-beijing.aliyuncs.com/releases/repo.html  -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install -U modelscope -i https://pypi.tuna.tsinghua.edu.cn/simple

错误2
transformers 版本低，升级到了transformers-4.44.2解决

Traceback (most recent call last):model = AutoModelForCausalLM.from_pretrained(File "/site-packages/modelscope/utils/hf_util.py", line 65, in from_pretrainedmodule_obj = module_class.from_pretrained(model_dir, *model_args,File "/site-packages/transformers/models/auto/auto_factory.py", line 441, in from_pretrainedconfig, kwargs = AutoConfig.from_pretrained(File "/site-packages/transformers/models/auto/configuration_auto.py", line 917, in from_pretrainedconfig_class = CONFIG_MAPPING[config_dict["model_type"]]File "/site-packages/transformers/models/auto/configuration_auto.py", line 623, in __getitem__raise KeyError(key)
KeyError: 'qwen2'

 pip install -U transformers -i https://pypi.tuna.tsinghua.edu.cn/simple