January 31, 20251 yr Could anyone advise me which videocard I need when I want to experiment with selhosting AI models (open webgui, chat GPT alternative). I have a an ASUSTeK COMPUTER INC. Pro WS C621-64L SAGE Series with now an Nvidia Quadro K2200 card (from an old AutoCAD desktop). If I ask a question now, it takes 20 seconds before I get an answer. My Unraid server is also used for many other services (5 VM's, 20 Dockers), so it will probably take a lot of money to make it just as fast as CHat GPT, but I want to spend only 500 euro for an GPU.
March 4, 20251 yr I am also looking at the same sort of thing for my server. Nivida gpus are better then amd for AI processing. Without a gpu, only use the smaller models (llama3.2, dolphin-llama3 etc) From what i can tell, its all about the VRAM of the gpu. You at bare minimum want something with 8GB vram, but ideally 12GB vram. the higher the cuda cores and tensors the better too which will reduce time to process, but thats where prices go up from a rtx 3060 to very high prices for rtx 4070. https://www.sportskeeda.com/gaming-tech/best-graphics-cards-12-gb-vram
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.