Suggestion for a brand new GPU card model

[obones]

Hello,

As I'm having quite a few issues with my "second hand" Tesla P4 compute card, I'm considering buying a brand new GPU board in the hopes it would work better.
I know that high end GPUs are just fine and many are mentioned in the benchmarks table, but I'm trying to aim at the 200€ mark and what I'm seeing are these:

GTX1630 with 4GB
GTX1650 with 4GB
RTX3050 with 8GB
RTX3060 with 8GB

I understand that 4GB may not be enough, but would any of these GPUs have enough computation power?
As "Black Friday" is approaching, there might be some deals on RTX3050/RTX3060 boards and I want to be ready for them.

Thanks for your advice.

[nikito]

I personally am running a GTX1070, I think they can be had for around $100 USD on ebay, and it has 8GB of ram. Using the Small model with beam size 2 (presently what us developers are using with excellent performance), my response times are around 300ms from end of speech to action done/TTS output. I would highly recommend avoiding 4GB cards as you will likely run into issues, since that is a very small amount of VRAM when considering the inference as well as TTS models required.

That said, anything at or above the GTX1070 will give excellent performance; If I had to pick from your above list I'd go for either of the 8GB cards. If you want to do more with GPU such as playingm around with LLMs you would want to shoot for a card with at least 12GB, preferable 16 or even 24GB of VRAM.

Hope this information helps! :)

[kristiankielhofner]

@nikito gave some good information but we'd also like to hear about the issues you're having with the Tesla P4. We have two developers using them exclusively with good results and many people in the community have reported the expected results as well. I myself just ordered another one for a SFF half-height "homelab" server configuration I'm working on.

[obones]

We already discussed this back in June (#87) and the situation hasn't improved since then, despite different updates on the driver side of things.
Maybe it's the docker integration, but I don't know how to run WIS outside docker on ArchLinux
But I'm more inclined to think it's the card itself that may have been "abused" by its former owner, hence the reason I'm asking about potential replacements. That said, I'm more than happy to discuss other things to try over in the other discussion