合并模型的时候显存不够,用的4090，24G

[mengxinru]

		  Traceback (most recent call last):

File "/root/autodl-tmp/chatGLM-6B-QLoRA-main/merge_lora_and_quantize.py", line 82, in
main(lora_path=args.lora_path,
File "/root/autodl-tmp/chatGLM-6B-QLoRA-main/merge_lora_and_quantize.py", line 56, in main
merged_model, lora_config = merge_lora(lora_path, device_map)
File "/root/autodl-tmp/chatGLM-6B-QLoRA-main/merge_lora_and_quantize.py", line 30, in merge_lora
model = PeftModel.from_pretrained(base_model, lora_path, device_map=device_map)
File "/root/miniconda3/lib/python3.10/site-packages/peft/peft_model.py", line 181, in from_pretrained
model.load_adapter(model_id, adapter_name, **kwargs)
File "/root/miniconda3/lib/python3.10/site-packages/peft/peft_model.py", line 372, in load_adapter
adapters_weights = torch.load(
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 1014, in load
return _load(opened_zipfile,
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 1422, in _load
result = unpickler.load()
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 1392, in persistent_load
typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 1366, in load_tensor
wrap_storage=restore_location(storage, location),
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 1299, in restore_location
return default_restore_location(storage, str(map_location))
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 381, in default_restore_location
result = fn(storage, location)
File "/root/miniconda3/lib/python3.10/site-packages/torch/serialization.py", line 279, in _cuda_deserialize
return obj.cuda(device)
File "/root/miniconda3/lib/python3.10/site-packages/torch/_utils.py", line 114, in _cuda
untyped_storage = torch.UntypedStorage(
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.00 MiB. GPU 0 has a total capacty of 23.65 GiB of which 2.06 MiB is free. Process 97728 has 23.64 GiB memory in use. Of the allocated memory 23.26 GiB is allocated by PyTorch, and 39.50 KiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF