A used Tesla P40 puts 24 GB of VRAM in a desktop for about a fifth of what a renewed RTX 3090 costs on Amazon right now. The catch is software, not speed. CUDA 13 can no longer build code for the P40, P100 or V100, and vLLM refuses all three.
You’ll find the best used datacenter GPUs for local AI below: the P40, P100, V100 and RTX A4000, plus the AMD MI50 and a new Intel card that competes on price. We benchmarked each NVIDIA card with llama.cpp v0.7.0 on rented machines in October 2026, and checked driver, Ollama and vLLM support against the current docs.
| Card | VRAM | Qwen3 8B decode | Qwen3 32B decode | Software ceiling | Price band |
|---|---|---|---|---|---|
| Tesla V100 32GB PCIe | 32 GB HBM2 | 109 t/s | 31.7 t/s | CUDA 12.x, driver 580 | about $860 renewed |
| Tesla P40 24GB | 24 GB GDDR5 | 47 t/s | 12.4 t/s | CUDA 12.x, driver 580 | about $420 renewed |
| Tesla P100 16GB PCIe | 16 GB HBM2 | 38 t/s | does not fit | CUDA 12.x, driver 580 | about $155 refurbished |
| RTX A4000 16GB | 16 GB GDDR6 | 72.5 t/s | does not fit | current CUDA, vLLM | about $1,045 renewed |
| Intel Arc Pro B60 24GB | 24 GB GDDR6 | not tested | not tested | Vulkan and SYCL, no CUDA | about $650 new |
Decode speeds are our llama-bench results with flash attention on, and the V100 numbers come from the SXM2 module we rented. Prices are Amazon US bands from October 22, so check the current price before you buy. Most are renewed server pulls, so read the return window.
Check software support before you buy
Age and a flood of decommissioned servers keep these cards cheap, and NVIDIA has now moved the oldest ones to its legacy list. The CUDA 13.0 release notes say offline compilation for Maxwell, Pascal and Volta was removed, and that toolkits through 12.x stay supported for building. Each card stands like this today:
| Card | Build with CUDA 13 | Newest driver branch | Ollama | vLLM | llama.cpp |
|---|---|---|---|---|---|
| P40, P100 | No, CUDA 12.x only | R580 LTS | Yes, driver 570 or newer | No | Yes, CUDA 12 build |
| V100 | No, CUDA 12.x only | R580 LTS | Yes, driver 550 or newer | No | Yes, CUDA 12 build |
| RTX A4000 | Yes | Current | Yes | Yes | Yes |
| MI50, MI60 | ROCm dropped it | In-kernel amdgpu | Vulkan only | No | Vulkan or community ROCm |
| Arc Pro B60 | Not CUDA | In-kernel xe | Vulkan | Yes, XPU backend | Vulkan or SYCL |
Three consequences follow from that table. Each one decides whether a cheap card fits the stack you already run:
- The vLLM GPU requirements now ask for compute capability 7.5, so the V100 (7.0) is out along with both Pascal cards.
- Ollama still lists all three Teslas, but its GPU docs ask for driver 570 or newer on compute capability 6.x.
- NVIDIA’s 595 and 615 Linux drivers list the P40, P100 and V100 only under the 580 legacy table.
A host left on an older driver does not fail loudly with Ollama. It falls back to the CPU, and the model just runs slowly, so install the 580 branch first with our NVIDIA driver and CUDA guide for Ubuntu.
Compare the cards on the same models
We rented one card of each type on vast.ai and built llama.cpp v0.7.0 from source against a CUDA 12 toolkit, with the architecture set per card. Each card ran the same GGUF files where they fit: Qwen3 8B and Qwen3 32B at Q4_K_M, and gpt-oss-20b in MXFP4. The 16 GB cards skipped the 32B model.
We ran this build and benchmark on the P40 after cloning the v0.7.0 tag of llama.cpp. The 61 is its compute capability, and the P100, V100 and A4000 used 60, 70 and 86. Run it with a CUDA 12 toolkit on the path, because CUDA 13 cannot build for the Teslas. The GGUF files came from the unsloth repos on Hugging Face, such as unsloth/Qwen3-32B-GGUF, downloaded into the working directory first:
git clone --depth 1 --branch v0.7.0 https://github.com/ggml-org/llama.cpp
cmake -S llama.cpp -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=61 -DCMAKE_BUILD_TYPE=Release
cmake --build build -j"$(nproc)" --target llama-bench
build/bin/llama-bench -m Qwen3-32B-Q4_K_M.gguf -ngl 99 -p 512 -n 128 -r 3 -fa 0,1On the P40, the 18.4 GiB model sat fully on the card. Decode settled at about 12 tokens per second:
| model | size | params | backend | ngl | fa | test | t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen3 32B Q4_K - Medium | 18.40 GiB | 32.76 B | CUDA | 99 | 0 | pp512 | 220.66 ± 0.31 |
| qwen3 32B Q4_K - Medium | 18.40 GiB | 32.76 B | CUDA | 99 | 0 | tg128 | 12.02 ± 0.01 |
| qwen3 32B Q4_K - Medium | 18.40 GiB | 32.76 B | CUDA | 99 | 1 | pp512 | 219.15 ± 0.24 |
| qwen3 32B Q4_K - Medium | 18.40 GiB | 32.76 B | CUDA | 99 | 1 | tg128 | 12.41 ± 0.00 |Below is the full set with flash attention on. Prompt processing (pp512) is how fast the card reads your prompt, and decode (tg128) is how fast it writes the answer:
| Card | Qwen3 8B pp512 / decode | gpt-oss-20b pp512 / decode | Qwen3 32B pp512 / decode | Idle draw | Peak draw |
|---|---|---|---|---|---|
| Tesla V100 SXM2 32GB | 2,909 / 109.2 | not run | 765 / 31.7 | 42 W | 291 W |
| Tesla P40 24GB | 915 / 47.0 | 1,418 / 62.7 | 219 / 12.4 | 11 W | 242 W |
| Tesla P100 PCIe 16GB | 683 / 38.4 | 524 / 65.2 | does not fit | 28 W | 211 W |
| RTX A4000 16GB | 2,733 / 72.5 | 3,138 / 114.8 | does not fit | 16 W | 140 W |
A few things in that table need context before you lean on it. The V100 we rented was an SXM2 module, not the PCIe card we link, and the PCIe version has the same 900 GB/s memory but a 250 W limit instead of 300 W. Its gpt-oss-20b download stalled twice on that host, so we have no number for it.
We also rented an RTX 3090 as a baseline, but that host had capped the card at 220 W and it decoded the 8B model slower than the A4000. We left it out. For a fair reference, the llama.cpp CUDA scoreboard lists a 3090 at about 158 tokens per second on Llama 2 7B Q4_0, against about 129 for a V100 and 55 for a P40.
Memory bandwidth explains most of the ranking. The V100’s HBM2 beats the newer A4000, and the A4000 beats the P40’s GDDR5. The P100 is the exception: it has twice the P40’s bandwidth but decoded the 8B model slower, likely because its older compute capability lacks the int8 dot-product instruction that llama.cpp’s quantized kernels use. Prompt processing on the P40 is about a third of the V100’s, so long RAG prompts feel slow on it.
Match the card to your budget and model
Start from the largest model you plan to keep loaded, then pick the cheapest card that holds it with room for context. Our VRAM sizing guide has the math for each model size.
VRAM per dollar is the number most buyers compare first. At the Amazon bands we checked, it works out like this:
| Card | Price band | Cost per GB of VRAM |
|---|---|---|
| Tesla P100 16GB (certified refurbished) | about $155 | about $10 |
| Tesla P40 24GB (renewed) | about $420 | about $17 |
| Tesla V100 16GB PCIe (renewed) | about $365 | about $23 |
| Tesla V100 32GB PCIe (renewed) | about $860 | about $27 |
| Intel Arc Pro B60 24GB (new) | about $650 | about $27 |
| RTX A4000 16GB (renewed) | about $1,045 | about $65 |
| RTX 3090 24GB (renewed) | about $1,980 | about $83 |
The shortage shows in the last row. A renewed 3090 costs almost five times a renewed P40 for the same 24 GB, which is why so many buyers put up with the P40’s speed.
Tesla V100 32GB PCIe: best used card overall
Of the cards here, the V100 is the fastest at decode, and the 32 GB version holds a 32B model at Q4 with room for a long context. On our rented module it wrote 31.7 tokens per second on Qwen3 32B, more than twice the P40. Prompt processing ran about three times faster than the P40 on both models, which matters for long prompts.
Buy it if you want 30B-class models at a comfortable reading speed and can live without vLLM. Skip it if you need vLLM or plan to keep upgrading CUDA, because Volta is frozen at 12.x. The 16 GB PCIe version runs about $365 when 8B to 14B models are enough.
Tesla P40 24GB: cheapest 24 GB card
Most buyers start with the P40, because it is the reason this market exists: 24 GB for about $420 renewed, against roughly $1,980 for a renewed RTX 3090. It fits a 32B model at Q4 that no 16 GB card can hold. It also idled at 11 W in our test, the lowest of the Teslas.
You pay for that price in speed. Decode on the 32B model ran at 12.4 tokens per second, which is readable but slow, and FP16 runs at 1/64 of the FP32 rate on the P40. Buy it for a 24 GB budget build or a pair for 48 GB. Skip it if you will fine-tune or use FP16-heavy engines.
Tesla P100 16GB PCIe: cheapest way in
At about $155, the P100 is the cheapest 16 GB card here, and its 16 GB is HBM2. It decoded gpt-oss-20b at 65.2 tokens per second, slightly faster than the P40, though it trailed the P40 on the dense 8B model. Prompt processing was the slowest in our test.
Buy it to try local models on a server you already own, with 8B to 20B models in mind. Skip it if you want 24 GB later, because two P100s cost about three quarters of a P40 and draw twice the power.
RTX A4000 16GB: easiest card to live with
Of the NVIDIA cards we tested, only the A4000 works like a normal desktop GPU, so you won’t need adapters or a fan. It is single slot, has a blower fan and four DisplayPort outputs, draws 140 W from one 6-pin connector, and runs current CUDA, so vLLM works on it. It decoded gpt-oss-20b at 114.8 tokens per second, the best of the three cards that ran it.
Price is where it loses. The renewed listing we link sits near $1,045, which buys a V100 32GB with money left over. Buy it for a small case, a shared workstation, or a stack built on vLLM. Skip it if VRAM per dollar is the goal.
Intel Arc Pro B60 24GB: new card at used prices
We did not benchmark the B60, but it changes the math. It is a new 24 GB card with a warranty and display outputs for about $650, close to the VRAM price of a used V100 32GB. The llama.cpp Vulkan scoreboard shows about 69 tokens per second on Llama 2 7B, above a P40.
You give up CUDA, so anything that only ships CUDA builds will not run. Buy it if llama.cpp or Ollama on Vulkan covers your models. Our Arc Pro B70 and Radeon AI Pro R9700 comparison covers the bigger Intel card.
Know what you give up with an MI50
AMD’s Instinct MI50 32GB shows up in every used-GPU thread, because 1 TB/s of HBM2 decodes about twice as fast as a P40 on the llama.cpp Vulkan scoreboard. The ROCm 6.4 system requirements mark its gfx906 chip as unsupported, and every ROCm release since keeps it that way. Ollama’s ROCm list starts at the MI100.
You can still run llama.cpp on it through Vulkan, and community ROCm builds for gfx906 are maintained. We did not find a credible MI50 32GB listing on Amazon US, only one 16 GB card from an unbranded seller, so we cannot recommend a place to buy one.
Fit a passive card into a desktop
Every Tesla here is built for a server chassis that pushes air through it. In a desktop, plan for these before the card arrives:
- Power: an EPS to dual PCIe 8-pin adapter, about $13
- Cooling: a fan shroud plus a 97×33 mm blower
- Firmware: Above 4G Decoding turned on
- Display: an iGPU or a second card
- Mixing: no Blackwell card in the same box
Each line needs a short explanation. The P40, P100 and V100 PCIe take one CPU 8-pin (EPS) plug, not a PCIe plug, and you should never force a PCIe lead into it. They have no fan of their own. The shroud kit we link, about $20, is only the shroud and bolts, so buy a 97×33 mm blower separately and run it from a motherboard fan header so the BIOS can control its speed. Check the blower’s rated current first: many of these draw more than the roughly 1 A a header supplies, and in that case a SATA-powered fan hub is the safer feed. The P40 maps a 32 GB BAR and the P100 and V100 map 16 GB, so without Above 4G Decoding the board may not boot, or the driver fails to start the card.
None of the Teslas has a display output, so you’ll keep an iGPU or a second card for the screen. Pascal and Volta also need NVIDIA’s proprietary kernel module while Blackwell needs the open one, and only one can load, so you can’t pair a Tesla with any Blackwell card, including RTX 50 and RTX PRO Blackwell.
Idle power changes once a model is loaded. Our readings were taken before loading anything, and owners report Tesla cards sitting in P0 at about 50 W while a model stays resident. The nvidia-pstated daemon drops them back to P8 when they’re idle.
For two or more cards, check the slot spacing and lane layout first. Our multi-GPU motherboard picks and PCIe lane guide cover both.
Choose by the model you want to keep loaded
For 30B-class models, the V100 32GB is the used card to buy, as long as you can stay on CUDA 12 and skip vLLM. The P40 is the budget 24 GB card, slow but cheap. The P100 is a cheap experiment, and the A4000 is for people who want a normal GPU and current CUDA. If a new card fits the budget, compare these against our best GPUs for local LLMs and the GPU price tracker before you order.
