Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
When running AWQ search foe Llama2 with transformers>=4.38.0, I find the bug below: File "/×××/llm-awq/awq/quantize/auto_scale.py", line 134, in _search_module_scale RuntimeError: The expanded size of the tensor (4608) must match the existing size (4096) at non-singleton dimension 3. Target sizes: [65, 32, 512, 4608]. Tensor sizes: [65, 1, 512, 4096] But earlier versions of transformers would not happen this. So I set transformers==4.36.2.
- Loading branch information