March 17, 20251 yr I have asked this not too long ago, however my budget has changed... so my GPU selection has changed. Due to the pricing being "close-ish" I am looking at the Quadro GV100 and the RTX 8000 GV100: 32GB Ram (HBM) RTX 8000 48GB RAM So I am looking at 2 things, Yes, I know the RTX 8000 has 16 GB more ram.. and that alone is almost a reason to go that direction. However, for AI workloads, does the HBM on the GV100 give benefit having a higher bandwidth and MUCH larger interface bus? Also the GV100 does have more Cuda and Tensor cores... I understand they are technically a generation older... not sure how much that matters. So put aside the memory size, what would be the "better" card to go with? I do plan to get a 2nd one eventually no matter what card I go with... so down the road I will have 64GB or 96GB of Vram...either way will be plenty. Would I assume correctly that the RTX 8000 would be the better card, or does the HBM give the edge to the GV100? Thanks!
March 17, 20251 yr Not an expert on AI, LLM or neural networks. But I do have some wicked Google Fu skills. And two very different queries ended up on this blog post. I also have 40 years of computer hardware design and semiconductor manufacturing experience, so I understand trying to compare memory size, memory type, Tensor cores, different generations of GPU chips, etc. The blog seems to prioritize the number of Tensor cores (performance) over memory bandwidth. My instinct is that this is correct. Edited March 17, 20251 yr by ConnerVT
March 17, 20251 yr Author 27 minutes ago, ConnerVT said: Not an expert on AI, LLM or neural networks. But I do have some wicked Google Fu skills. And two very different queries ended up on this blog post. I also have 40 years of computer hardware design and semiconductor manufacturing experience, so I understand trying to compare memory size, memory type, Tensor cores, different generations of GPU chips, etc. The blog seems to prioritize the number of Tensor cores (performance) over memory bandwidth. My instinct is that this is correct. I have come across that post in the past... there is a lot of technical talk that is above my head (and I thought I had decent understanding of things) So I take it the GV100 would be the "better" direction to go? As far as general performance due to having higher Tensor + Cuda core count? I understand due to how fast the Tensor cores are they are usually "idle" waiting on memory, so assumption would be a card with faster / better memory would be more efficient. Correct assumption? On the performance chart they have on the site, they show the V100 vs the RTX 8000... looks like the V100 is shown to have an edge. (basically the same as the GV100 if they are referring the the PCIe version of the V100) Edited March 17, 20251 yr by KooKoo102
March 17, 20251 yr That's where it gets fuzzy. The GV100 is a generation behind the newer RX8000. Which would explain the 10% lower Tensor Performance rating. But then, it has 10% more Tensor cores. HBM2 memory is inherently faster. Thus the higher memory bandwidth. But memory only comes into play if it is the bottleneck, and can't keep up with the computational speed of the cores. Who knows? Above my pay grade on that. The chart you posted with the benchmarks, if from a reputable source and comparable to the tasks you plan, may be the tie breaker.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.