-
At the moment it seems to only be using a fraction of the available CPU/RAM my computer has available (I have a high end CPU and a LOT of RAM). I would assume if I could increase the RAM usage 2 or 3 times than generation would scale accordingly. Is there a way to change how much resources KoboldCPP utilizes during generation? I set KoboldCPP's priority in task manager to "realtime" and my computer to "performance mode" and that seemed to give generation speed a small boost but not by much. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
You can set it to use more threads with the --threads parameter. |
Beta Was this translation helpful? Give feedback.
You can set it to use more threads with the --threads parameter.