Avieshek@lemmy.world to Technology@lemmy.worldEnglish · 3 months agoEdward Snowden slams Nvidia's RTX 50-series 'F-tier value,' whistleblows on lackluster VRAM capacitywww.tomshardware.comexternal-linkmessage-square101fedilinkarrow-up1373arrow-down195
arrow-up1278arrow-down1external-linkEdward Snowden slams Nvidia's RTX 50-series 'F-tier value,' whistleblows on lackluster VRAM capacitywww.tomshardware.comAvieshek@lemmy.world to Technology@lemmy.worldEnglish · 3 months agomessage-square101fedilink
minus-squareEager Eagle@lemmy.worldlinkfedilinkEnglisharrow-up48·3 months agoI bet he just wants a card to self host models and not give companies his data, but the amount of vram is indeed ridiculous.
minus-squareJeena@piefed.jeena.netlinkfedilinkEnglisharrow-up25·3 months agoExactly, I’m in the same situation now and the 8GB in those cheaper cards don’t even let you run a 13B model. I’m trying to research if I can run a 13B one on a 3060 with 12 GB.
minus-squareThe Hobbyist@lemmy.ziplinkfedilinkEnglisharrow-up15·3 months agoYou can. I’m running a 14B deepseek model on mine. It achieves 28 t/s.
minus-squareJeena@piefed.jeena.netlinkfedilinkEnglisharrow-up6·3 months agoOh nice, that’s faster than I imagined.
minus-squarelevzzz@lemmy.worldlinkfedilinkEnglisharrow-up4·3 months agoYou need a pretty large context window to fit all the reasoning, ollama forces 2048 by default and more uses more memory
minus-squaremanicdave@feddit.uklinkfedilinkEnglisharrow-up4·3 months agoI’m running deepseek-r1:14b on a 12GB rx6700. It just about fits in memory and is pretty fast.
I bet he just wants a card to self host models and not give companies his data, but the amount of vram is indeed ridiculous.
Exactly, I’m in the same situation now and the 8GB in those cheaper cards don’t even let you run a 13B model. I’m trying to research if I can run a 13B one on a 3060 with 12 GB.
You can. I’m running a 14B deepseek model on mine. It achieves 28 t/s.
Oh nice, that’s faster than I imagined.
You need a pretty large context window to fit all the reasoning, ollama forces 2048 by default and more uses more memory
I’m running deepseek-r1:14b on a 12GB rx6700. It just about fits in memory and is pretty fast.