Build a Dual RTX 3090 Rig for Local LLMs: Parts and 70B Benchmarks

Can two used RTX 3090s still run a 70B model fast enough to use every day, or has the trick aged out? The pair gives you 48 GB of VRAM for less than one RTX 5090, and a 5090 cannot hold a 70B at 4-bit at all. The usual warning is that two cards run no faster than one, and that warning only holds for the default split mode.

Original content from computingforgeeks.com - post 175867

We rented a dual RTX 3090 host on vast.ai and measured Llama 3.3 70B and Qwen3 32B in llama.cpp and vLLM, one card against two. This guide gives those numbers first, then the dual RTX 3090 build that fits two 3-slot cards without a riser, then the exact commands we ran. We ran everything on October 19, 2026 with llama.cpp build b11741, vLLM 0.31.0 and NVIDIA driver 580.95.05 in an Ubuntu 24.04 container, so pin those or newer if you want to compare numbers.

Dual RTX 3090 benchmark results

Every row below is our own run on the rented rig, with both cards capped at 290 W by the host. Generation speed is what you feel in a chat window, and prompt processing is how fast the rig reads in a long prompt or a document you paste.

Model and engine	Cards and split	Prompt tok/s	Generation tok/s	Peak VRAM per card
Llama 3.3 70B Q4_K_M, llama.cpp	2, layer split (default)	577	19.5	20,876 MiB
Llama 3.3 70B Q4_K_M, llama.cpp	2, tensor split	763	31.9	20,776 MiB
Llama 3.3 70B Q4_K_M, llama.cpp, 8K context filled	2, tensor split	656	29.9	22,074 MiB
Qwen3 32B Q4_K_M, llama.cpp	1	1,198	35.3	19,214 MiB
Qwen3 32B Q4_K_M, llama.cpp	2, tensor split	1,578	59.8	9,976 MiB
Llama 3.3 70B AWQ, vLLM, 1 user	2, tensor parallel	n/a	31.3	95% reserved
Llama 3.3 70B AWQ, vLLM, 8 users	2, tensor parallel	n/a	148.8 total	95% reserved

On the 70B, tensor split generated 1.64 times faster than the default layer split, and two cards ran the 32B 1.69 times faster than one. Layer split puts half the layers on each card and runs them one after the other, so only one card works at a time. Tensor split puts part of every layer on both cards, which is why the second card adds speed instead of only adding memory, as long as you pass the flag yourself.

How we tested the rig

We filtered vast.ai for hosts with two RTX 3090s, and the third host we tried, with a 0.999 reliability score, produced every number here. The first two failed before any benchmark ran, one on a download that slowed to a crawl and one on an image that never finished loading. The working rig had this hardware:

• 2x RTX 3090 24 GB, capped at 290 W each by the host
• PCIe Gen3 x16 per card, about 16 GB/s each way
• No NVLink bridge, and no direct peer-to-peer access
• Xeon E5-2680 v4 with 28 vCPUs on an X99 board
• 62 GB of system RAM

Gen3 x16 moves about the same data as Gen4 x8, which is what each card gets on an AM5 board running two cards, so the bus results carry over to the build below. The PCIe lanes guide covers that maths. The topology check showed the two cards talking through the CPU host bridge:

nvidia-smi topo -m

PHB means traffic between the cards crosses the CPU host bridge rather than a bridge or NVLink:

	GPU0	GPU1	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	PHB	0-27	0		N/A
GPU1	PHB	 X 	0-27	0		N/A

For your own rig, the power cap matters more than the old Xeon. Stock RTX 3090s run at 350 W, and the host blocked any change to the limit, so we could not measure the trade-off between power and speed. If you run the cards at stock power at home, you should match or beat these figures, most of all on prompt processing. The whole session, including the two failed hosts, cost $1.16 in rental time.

Step 1: Pick the RTX 3090 cards

Your choice of card decides the rest of the build, because its width sets the slot spacing and its power plugs set the power supply. We picked the Founders Edition for both builds because it had the deepest renewed stock on Amazon, eight units, and takes two 8-pin plugs through the adapter NVIDIA includes.

The NVIDIA spec page lists the Founders Edition as a 3-slot card, 313 mm long, rated at 350 W, with two 8-pin PCIe plugs feeding a 12-pin adapter that comes in the box. On October 20, Amazon showed the renewed Founders Edition at $1,899.99 with only a few units left. The MSI Ventus 3X OC was $1,829.97 renewed with two left and the ASUS TUF OC was $1,999.99, and both use two 8-pin plugs as well.

Used cards cost less than any of those. The ResalePrices tracker put a fair used price at $1,456 to $1,558 on October 19, up 22.5 percent over 90 days, which takes about $680 to $890 off either build below. Cards from private sellers usually come without the Amazon Renewed Guarantee, so run a long load test and check the fans before any return period ends.

Step 2: Pick a board with 4-slot spacing

If you stack two 3-slot cards, you need four slot positions between the top of one card and the top of the next, or the top card sits against the lower one with no gap for air. The usual board for this kind of build, the ASUS ProArt X870E-Creator, puts its two x16 slots three positions apart, which is the problem this step solves.

The Taichi Lite manual shows two CPU-fed PCIe 5.0 x16 slots that run x8/x8 with two cards. Measured from the layout diagram, they sit about 78 to 80 mm apart, which is four slot positions. A Level1Techs thread on the same problem recommends it for that spacing with Gen5 x8/x8 from the CPU. The MSI MEG X870E GODLIKE and Gigabyte X870E AORUS XTREME AI TOP share the layout at a higher price, and our multi-GPU motherboard guide compares them. It cost $299.99 on October 20, about $200 less than the ASUS ProArt X870E-Creator. If you look at other boards, check three things in the manual first:

• Both x16 slots wired to the CPU, x8/x8
• Four slot positions between them
• No M.2 slot sharing lanes with slot 2

A board that misses any of those either starves the second card or needs a riser to fit it.

Our local AI workstation build uses that ProArt, and its second slot drops to x4 when the M.2_2 slot holds a drive. That build needs a riser for the second card, and with the Taichi Lite you do not. The 4-slot spacing also matches the GeForce RTX 3090 NVLink bridge, which only comes in 4-slot width, but the only Amazon listing for that bridge was used at $1,995.98, and our rig scaled without it.

Step 3: Add the CPU, memory and storage

We did not compare CPUs, and with every layer offloaded to the cards the spec that matters for this build is PCIe lanes. Every Ryzen 9000 chip has the same 24 usable PCIe 5.0 lanes. The Ryzen 7 9700X at $309.99 is enough for the budget build. The Ryzen 9 9900X was only $16 more on October 20 from a third-party seller, so if that gap lasts when you buy, the extra four cores cost almost nothing.

Memory is the expensive surprise you will hit in 2026. We picked a 64 GB kit for the budget build, because the largest model file we loaded, the 70B GGUF, is 42.5 GB. The DDR5 price spike explains why that kit now costs $999.99.

For the recommended build, the Crucial 128 GB (2×64 GB) DDR5-5600 kit was $2,399.99 from a third-party seller with few left. Buy two large sticks rather than four small ones, because AMD rates four DIMMs at DDR5-3600 against DDR5-5600 for two. The DDR5 buying guide compares more kits.

Storage fills faster than you expect. These are the four models we downloaded for the tests:

• Llama 3.3 70B Q4_K_M GGUF, 42.5 GB
• Llama 3.3 70B AWQ, 39.8 GB
• Qwen3 32B Q4_K_M GGUF, 19.8 GB
• Qwen3 32B AWQ, 19.3 GB

Together they take about 121 GB, so a 2 TB drive gives you room for a working set of models plus a few quant variants of each.

The Samsung 990 PRO 2TB was $409.99 on October 20. For cooling, the Thermalright Peerless Assassin 120 SE at $31.41 handles a 65 W 9700X, while the recommended build uses the quieter Noctua NH-D15 G2 at $159.95.

At 168 mm tall, the NH-D15 G2 fits the case in the next step, which takes coolers up to 195 mm. An air cooler also saves finding room for a radiator in a case where two cards already take most of the space.

Step 4: Size the power supply and case

Plan for more than the rated 350 W per card. Igor’s Lab measured spikes of 1 to 10 ms on the Founders Edition that can trip a power supply, and recommends 460 to 500 W of capacity per card when the card is overclocked to its 400 W limit. That budget is conservative for stock cards, and it puts two cards at 920 to 1,000 W before you add the CPU and the board.

The Corsair HX1500i (2025) at $319.99 ships five 8-pin PCIe cables, per the Corsair HXi manual, which covers two cards with two plugs each and leaves one spare. Corsair rates it at 1500 W on 115 to 240 V and 1200 W only on 100 to 115 V mains, such as the 100 V supply in Japan, so a North American 120 V outlet gets the full 1500 W.

The HX1200i (2025) at $279.99 has the same five cables and is the floor for the budget build. Two cards with three 8-pin plugs each need six cables, so pair those with the Seasonic PRIME PX-1600 at $496.92 instead.

Most of the room you need goes to the bottom card. Measured from the manual diagram, the second x16 slot on the Taichi Lite sits at the bottom slot position, so a 3-slot card there covers expansion positions seven to nine, and an ordinary 7-slot ATX case cannot close around it.

The Phanteks Enthoo Pro 2 Server Edition has 11 expansion slots, room for E-ATX boards and a 195 mm cooler limit. The glass version was $189.99 and the closed-panel version $179.99 on October 20. We have not measured temperatures or noise in this case with two Founders Editions inside, so be ready to run the front intake fans at full speed under long jobs.

Dual RTX 3090 build cost

Whichever build you choose, the cards and the board stay the same. The recommended build spends the difference on 128 GB of memory, the larger power supply and the quieter cooler:

Part	Budget build	Recommended build
Graphics cards	2x RTX 3090 FE renewed, $3,799.98	Same, $3,799.98
CPU	Ryzen 7 9700X, $309.99	Ryzen 9 9900X, $326.00
Motherboard	ASRock X870E Taichi Lite, $299.99	Same, $299.99
Memory	Corsair 64 GB (2×32) DDR5-6000, $999.99	Crucial 128 GB (2×64) DDR5-5600, $2,399.99
Power supply	Corsair HX1200i (2025), $279.99	Corsair HX1500i (2025), $319.99
Case	Enthoo Pro 2 Server, closed, $179.99	Enthoo Pro 2 Server, glass, $189.99
SSD	Samsung 990 PRO 2TB, $409.99	Same, $409.99
CPU cooler	Thermalright PA120 SE, $31.41	Noctua NH-D15 G2, $159.95
Total	$6,311.33	$7,905.88

We pulled these Amazon US prices on October 20, 2026, and the memory and renewed cards move every week, so check the live price before you buy. With two used cards at the tracker price, the budget build drops to about $5,420 to $5,630. That is below the $6,500 to $7,300 our 3090 vs 5090 comparison lists for one RTX 5090 card, and well under the roughly $10,000 workstation build linked above.

Step 5: Run a 70B in llama.cpp

With the rig assembled and the NVIDIA driver and CUDA toolkit installed, add the build tools and the Hugging Face CLI. Our rented container already had both, so these two lines come from the Ubuntu packages and the Hugging Face CLI guide rather than from our run:

sudo apt install -y build-essential cmake git
curl -LsSf https://hf.co/cli/install.sh | bash

Then build llama.cpp with CUDA for the Ampere cards. We pinned the build tag we tested, because upstream still marks tensor split as experimental:

git clone --depth 1 --branch b11741 https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF
cmake --build build --config Release -j $(nproc) --target llama-bench llama-cli llama-server

On the 28-thread Xeon the build took 400 seconds. It also printed a warning that NCCL was not found, so our tensor split runs used the internal multi-GPU path in llama.cpp, and we did not measure a build linked against NCCL. Log in to Hugging Face before downloading, because the Hub gives unauthenticated requests lower rate limits:

hf auth login
HF_XET_HIGH_PERFORMANCE=1 hf download bartowski/Llama-3.3-70B-Instruct-GGUF Llama-3.3-70B-Instruct-Q4_K_M.gguf --local-dir ~/models/gguf-70b

With high-performance Xet transfers on, which that variable enables for you, the 42.5 GB file took 410 seconds on the rented host. Once you have it, benchmark both cards with tensor split, offloading every layer:

CUDA_VISIBLE_DEVICES=0,1 ./build/bin/llama-bench -m ~/models/gguf-70b/Llama-3.3-70B-Instruct-Q4_K_M.gguf -ngl 99 -sm tensor -p 512 -n 128 -r 3

Our run reported 31.93 tokens per second for generation across three repetitions:

| model                          |       size |     params | backend    | ngl |     sm |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | -----: | --------------: | -------------------: |
| llama 70B Q4_K - Medium        |  39.59 GiB |    70.55 B | CUDA       |  99 | tensor |           pp512 |        763.09 ± 0.64 |
| llama 70B Q4_K - Medium        |  39.59 GiB |    70.55 B | CUDA       |  99 | tensor |           tg128 |         31.93 ± 0.09 |

Running the same command with -sm layer gave 19.48 tokens per second, which is the number behind the claim that two cards run no faster than one. For a chat server, the same –split-mode flag works on llama-server, as listed in the llama-server README, and our llama.cpp guide covers the server setup.

Step 6: Serve several users with vLLM

llama.cpp suits one user at a time. When a household or a small team shares the rig, vLLM batches requests across both cards, and the GGUF vs AWQ comparison explains why it uses an AWQ checkpoint instead of the GGUF file. After installing vLLM, download the AWQ model and start the server across both cards:

HF_XET_HIGH_PERFORMANCE=1 hf download casperhansen/llama-3.3-70b-instruct-awq --local-dir ~/models/awq-70b
vllm serve ~/models/awq-70b --served-model-name llama-3.3-70b-awq --tensor-parallel-size 2 --max-model-len 4096 --gpu-memory-utilization 0.95 --max-num-seqs 16 --port 8000

From launch to ready the server took about four minutes, and you should read two lines in the log. The first says vLLM fell back from its own all-reduce to NCCL because the cards had no direct peer-to-peer path:

(Worker pid=8099) WARNING 10-19 21:33:00 [custom_all_reduce.py:280] Custom allreduce is disabled because your platform lacks GPU P2P capability or P2P test failed. To silence this warning, specify disable_custom_all_reduce=True explicitly.

You can ignore that warning on a rig like this, because the benchmark below still scaled. The second line shows how much context the remaining VRAM can hold once the 70B weights are loaded:

(EngineCore pid=7974) INFO 10-19 21:35:10 [kv_cache_utils.py:2464] GPU KV cache size: 20,720 tokens, Maximum concurrency for 4,096 tokens per request: 5.06x

You get room for about five conversations of 4,096 tokens each at once, which suits a household but not a team running long documents. We measured throughput with the vLLM benchmark client, first with one user and then with eight. The client needs the bench extras, which install with pip install “vllm[bench]”:

vllm bench serve --backend vllm --base-url http://127.0.0.1:8000 --model llama-3.3-70b-awq --tokenizer ~/models/awq-70b --dataset-name random --random-input-len 512 --random-output-len 256 --num-prompts 8 --max-concurrency 1 --ignore-eos --seed 42

With one user, the 70B generated 31.25 tokens per second with a median of 721 ms to the first token:

Output token throughput (tok/s):         31.25     
Peak output token throughput (tok/s):    35.00     
Median TTFT (ms):                        721.13    
Median TPOT (ms):                        29.29     

With eight users and 32 prompts, the total rose to 148.81 tokens per second, while the median time to first token grew to 2.9 seconds. On Qwen3 32B AWQ the second card mattered even more, as these runs show:

Qwen3 32B AWQ in vLLM	1 card	2 cards, tensor parallel
Output tok/s, 1 user	37.8	57.4
Output tok/s, 8 users	201.1	290.8
Median time to first token, 8 users	2.3 s	1.4 s
Context cache (tokens)	11,536	97,632

With the second card, the 32B got 8.5 times the context cache, because each card now holds only half the weights. For mid-size models, that is the real reason you add a card: more users and longer prompts at once, not only faster single replies.

What broke on the test rig

Four things failed or behaved oddly during the runs, and each one changes how you should set up your own rig:

• -sm row would not load on CUDA
• nvidia-smi -pl was blocked on the rental host
• Single-card vLLM failed on one of the two cards
• Two hosts failed before any benchmark ran

You will see the row split mode listed in llama-bench help, but on this build it stops at model load with a split buffer error. It matches an open llama.cpp issue, so use -sm tensor on NVIDIA cards:

llama_model_load: error loading model: device CUDA0 does not support split buffers

Inside the vast.ai container we could not change the power limit, and both hosts we tried refused it. On a machine you own, sudo nvidia-smi -pl 280 sets a lower cap inside the range each card reports, which was 100 to 365 W on one of our cards and 100 to 350 W on the other. We could not measure the effect on speed, and this is the error the rental host returned:

Failed to set power management limit for GPU 00000000:02:00.0: Insufficient Permissions

Single-card vLLM on GPU0 failed three times with cuBLAS errors, while the same settings ran on GPU1, and both cards worked together under tensor parallel. We did not find the cause, so treat it as a reason to leave some VRAM headroom rather than as a limit of 24 GB cards. If you rent before you buy, the two failed hosts are the lesson: filter for reliability and download speed before you pay for GPU time.

Dual 3090 vs a 5090 and Strix Halo

The L40S row is our own Ollama run, while the DGX Spark and Strix Halo rows are published third-party runs cited in our comparison posts. They use different engines and quants, so read the gaps as rough rather than exact:

Machine	70B-class generation	Source
2x RTX 3090, llama.cpp tensor split	31.9 tok/s (Llama 3.3 70B Q4_K_M)	This test
2x RTX 3090, llama.cpp layer split	19.5 tok/s	This test
One L40S 48 GB	16 tok/s (Llama 3.3 70B Q4_K_M, Ollama)	VRAM guide
DGX Spark 128 GB	4.4 tok/s (Llama 3.1 70B Q4_K_M, Ollama)	DGX Spark vs Strix Halo
Strix Halo 128 GB	4.1 tok/s (Qwen2.5 72B Q5_K_M, ROCm)	mini PC guide
One RTX 5090 32 GB	Cannot hold a 70B at Q4_K_M	3090 vs 5090

On a dense 70B, the pair runs about twice as fast as one L40S and roughly seven times faster than the 128 GB unified-memory machines. A single 5090 is still the faster card for models that fit in 32 GB, and Strix Halo wins on capacity, power and noise for large mixture-of-experts models. If you want a dense 70B at reading speed for a few users, two RTX 3090s on a 4-slot board are the cheapest machine in this comparison that gets there.