Best CUDA Books for GPU Programming, With Code Tested

Programming Massively Parallel Processors, 5th edition, is the CUDA book to read first if you want to understand how a GPU actually runs your code. The rest of the best CUDA books split by what you write every day: modern C++, Python with Numba, or the attention and GEMM kernels inside an LLM.

Original content from computingforgeeks.com - post 178979

Old CUDA books are cheap and still recommended on forums, so we cloned every companion repository we could find and compiled the code with nvcc from two toolkits. We checked both CUDA 13.4 and CUDA 12.9 in October 2026 on Ubuntu 24.04, compile-only with no GPU attached, so the results below measure code rot, not speed.

Which CUDA books are worth buying in 2026?

Eight books made the cut. The code column counts the sample programs (make targets, for the Fregly repository) that compiled with CUDA 13.4 out of the ones we tried, using each repository’s own build files where they worked:

Book	Best for	Language	Code on CUDA 13.4	Pages	Published
Programming Massively Parallel Processors, 5th ed.	How GPUs run code	CUDA C	No official repo	680	2026
GPU Programming with C++ and CUDA	C++ developers	C++	6 of 7	270	2025
GPU-Accelerated Computing with Python 3 and CUDA	Python developers	Python	Not built (notebooks)	534	2026
CUDA for LLMs	Transformer kernels	CUDA C++	8 of 11	425	Preorder
AI Systems Performance Engineering	Tuning AI workloads	CUDA, PyTorch, Triton	8 of 9	1,060	2025
Programming in Parallel with CUDA	Scientists and engineers	CUDA C++	8 of 8 with header paths set	395	2022
Parallel and High Performance Computing	HPC beyond one GPU	C, MPI, OpenMP, CUDA	2 of 2 with its simple Makefiles	704	2021
The CUDA Handbook	A desk reference	CUDA C	Not built	528	2013

Failures that only meant a library was missing from our minimal toolkit (PyTorch headers, NCCL, CUTLASS) are counted as failures here but explained per book, because they will build on a full workstation. Print prices ran from about $35 to $90 on October 28, and book prices move weekly, so check the current price before you order.

How did we test the CUDA books?

We ran an Ubuntu 24.04 container with gcc 13.3 and installed only the compiler pieces from NVIDIA’s apt repository: nvcc, the runtime headers, CCCL (Thrust, CUB and libcu++), cuBLAS and cuRAND. That is what a reader gets from a normal toolkit install on a workstation, minus the GPU driver. Where a book had a repository, we cloned it at its latest commit, and the checks were the same for each one:

• The repository is the one the publisher or author links, or its provenance is noted.
• The build files name a GPU architecture the current toolkit still accepts.
• An early sample, such as vector add, compiles without edits.
• A later chapter using streams, textures, cuBLAS or warp shuffles compiles too.
• Any failure on 13.4 is rerun on 12.9 to see whether CUDA 13 caused it.

The architecture check matters more than it sounds. CUDA 13 dropped every GPU older than Turing from the compiler, so a Makefile that still says sm_52 or sm_60 now stops before it reads a line of your code. Ask your own toolkit which targets it supports:

nvcc --list-gpu-arch

On CUDA 13.4 the list starts at compute_75, which is Turing (RTX 20 series and T4):

compute_75 compute_80 compute_86 compute_87 compute_88 compute_89 compute_90 compute_100 compute_110 compute_103 compute_120 compute_121 compute_107 

CUDA 12.9 still starts at compute_50 (Maxwell). If your card or your book’s build files sit between those two points, stay on a 12.x toolkit for that book or change the architecture flag. None of this tells you how fast a kernel runs, and we make no speed claims; if you need a card to run the code on, our GPU buying guide for local LLMs ranks them by VRAM and price.

1. Programming Massively Parallel Processors, 5th Edition

Hwu, Kirk and El Hajj teach the GPU execution model first and CUDA syntax second, which is why this book is still the one most courses assign:

The 5th edition is a 2026 release (Amazon lists June 17) at 680 pages across 25 chapters. You learn threads, memory, tiling and occupancy on small examples, then the book walks through the parallel patterns every GPU programmer reuses: convolution, stencil, histogram, reduction, scan, sort and sparse matrix work. The contents add chapters on filtering, wavefront algorithms, the multi-GPU API and advanced matrix multiplication optimizations, which is the material that was missing when people used the 4th edition to learn how GEMM kernels get fast.

There is no official companion repository, so there was nothing to compile. That is less of a problem than it sounds: the code in the book is short and written for you to type and change, and the concepts do not age with the toolkit. For most people learning CUDA from scratch, this is the one book to own. Before you buy, check that:

• You can read C code with pointers and arrays comfortably.
• You are willing to write the exercises yourself rather than clone them.
• You want the why behind the code, not a cookbook.

Skip it if you only work in Python and never plan to write a kernel, because the whole book is CUDA C. It was the most expensive pick at about $90 in paperback on October 28.

2. GPU Programming with C++ and CUDA

Paulo Motta’s book is the shortest route from modern C++ to working CUDA code, and it builds with CMake the way your own projects do:

Packt published it in August 2025 at 270 pages. It moves from a hello-world kernel through GPU architecture, parallel algorithms and performance strategies to streams, and it ends with exposing GPU code to Python. The repository lists CUDA Toolkit 12, Nsight Compute, CMake and Docker as its software, and its build files target sm_75 or the native card.

Six of the seven samples we tried compiled on CUDA 13.4, and all seven compiled on 12.9. The one failure is a good preview of what CUDA 13 changes: the chapter 10 Thrust example sets CMAKE_CXX_STANDARD 14, and the CCCL headers in CUDA 13 now refuse anything older than C++17:

/usr/local/cuda-13.4/targets/x86_64-linux/include/cccl/thrust/detail/config/cpp_dialect.h:61:6: error: #error Thrust requires at least C++17. Define CCCL_IGNORE_DEPRECATED_CPP_DIALECT to suppress this message.

The message says what to change: build that chapter as C++17. Skip this book if you have never written C++ templates or used CMake, because it assumes both from page one. It cost about $45 in paperback.

3. GPU-Accelerated Computing with Python 3 and CUDA

Python developers who want GPU speed without leaving Python should start here rather than with a C book:

Niels Cautaerts and Hossein Ghorbanfekr published it with Packt in March 2026 at 534 pages. The companion repository sets up one Pixi environment with the CUDA toolkit, the NVIDIA profiler, Numba and JAX, so you write kernels in Python and profile them with the same tools a C++ developer uses. It is one of three 2026 books on this list, and among the cheapest at about $36 in paperback.

We did not score this one in the compile test, because its code is Python notebooks that need a GPU to execute rather than nvcc builds. Skip it if your goal is to read the CUDA C++ inside PyTorch or vLLM, since Numba hides exactly the parts you would need to recognize there.

4. CUDA for LLMs

Elliot Arledge’s Manning book is the only one here written around the kernels that LLMs actually spend their time in:

It started life as CUDA for Deep Learning and was renamed. Manning’s book page lists 56 kernels, Ampere, Hopper and Blackwell features, Flash Attention, profiling with Nsight Compute and a PyTorch C++ extension pipeline, and says it is for engineers who are comfortable with C and C++. The print edition is still a preorder: Amazon showed a December 2026 date on October 28, while the early-access ebook is on Manning now.

Its repository, infatoshi/book.cu, had a commit on October 15, 2026, and its README says appendix A installs CUDA 13. Eight of the eleven samples compiled on both toolkits, from vector add through naive GEMM and softmax to an MNIST network, AWQ quantization and pipeline parallelism. The three that failed wanted a library we had not installed: the tensor parallel chapter needs nccl.h from NCCL, and the FP4 and Hopper GEMM samples need the CUTLASS headers. Several other chapters build PyTorch extensions, so plan on a full PyTorch install as well.

Nothing failed because of the toolkit itself, which is what you want from a 2026 book. Skip it if you need a printed copy this month or if your GPU is older than Hopper, since the tensor core chapter’s Makefile defaults to sm_90a, so Ampere and Ada cards need flag edits there. Readers coming from LLM engineering books will find this is where the serving stack meets the hardware. The preorder price was about $67.

5. AI Systems Performance Engineering

Chris Fregly’s O’Reilly book is for the engineer who has to make training or inference on NVIDIA hardware cheaper, not the one learning what a warp is:

At 1,060 pages it covers CUDA kernels, PyTorch and torch.compile, OpenAI Triton, and distributed training strategies from data parallel to mixture of experts. The inference chapters cover vLLM, SGLang, TensorRT-LLM and NVIDIA Dynamo, and the book closes with a checklist of more than 200 tuning items. If you already run models with vLLM or llama.cpp and want to know where the time goes, this is the book.

Its repository was the most active we cloned, with a commit on October 27, 2026, and its shared build file assumes a current Blackwell setup:

• The architecture targets are sm_100a, sm_103a, sm_120 and sm_121.
• The default toolkit version is CUDA 13.0.
• The shared flags use -std=c++17, and the TMA pipeline example builds with -std=c++20.

Eight of nine samples compiled on both toolkits, including the cluster, distributed shared memory and TMA pipeline examples. The ninth needed PyTorch’s ATen headers. Skip it as a first CUDA book, and expect to edit the architecture flags if your card is an RTX 3090 or 4090. The paperback was about $69 against a $100 list price.

6. Programming in Parallel with CUDA

Richard Ansorge wrote this practical guide for people who need CUDA to finish their own computations, and the code is still maintained four years after printing:

Cambridge University Press published it in 2022 as a 395-page hardcover. The RichardAns/CUDA-Programs repository has a Ver2026 branch with commits from this year, and its README says it carries build files for Visual Studio 2022 and CUDA 13.1. Every Makefile targets sm_86, which is an RTX 30 series card.

All eight samples compiled on both toolkits, from the chapter 1 sum through a cuBLAS matrix multiply to the chapter 11 samples. The stock Makefile failed first, though, with fatal error: cx.h, because it expects two header paths you set yourself:

• CX_ROOT points at the repository’s own include directory.
• NVCUDASAMPLES_ROOT points at the Common folder from NVIDIA’s cuda-samples repository.
• CUDA_PATH matches the toolkit you installed.

Skip it if you want AI kernels, since the examples lean toward simulation, image processing and numerical work. Amazon listed it as scarce stock at about $60 on October 28.

7. Parallel and High Performance Computing

Robert Robey and Yuliana Zamora cover the whole HPC toolbox, and the GPU chapters are one part of a bigger argument about where parallelism belongs:

Manning published the 704 pages in 2021, covering OpenMP, MPI and GPU programming with CUDA and other models side by side. Read it if your code runs on a cluster and you have to decide which loops go to the GPU and which stay on the CPU. The print edition is black and white.

The CUDA chapter’s code dates from December 2020. We built its two samples, a stream triad and a sum reduction, several ways. The simple Makefiles built both on both toolkits, but the CMake build of the sum reduction sample failed on 13.4 because the old FindCUDA module asks for an architecture CUDA 13 no longer has:

nvcc fatal   : Unsupported gpu architecture 'compute_53'

Use the Makefile.simple files in that repository on CUDA 13 and the samples compile. Skip the book if you only care about one GPU in one box, because most of its pages are about everything else. It sold for about $51 against a $70 list price.

8. The CUDA Handbook

Nicholas Wilt’s book is the one older title we would still put on a shelf, as a reference rather than a course:

Addison-Wesley published it in June 2013 at 528 pages. That puts it four years before Volta, so tensor cores, cooperative groups and the _sync warp intrinsics are not in it. What it gives you is a careful account of the CUDA platform from someone who worked on it, which still explains why memory transfers, streams and the driver behave the way they do.

We did not test its code, and samples from 2013 are likely to hit the same architecture and API removals the next section shows. Skip it unless you already know modern CUDA and want the background. It was about $35 against a $60 list price, with scarce stock.

Which older CUDA books no longer compile?

CUDA by Example and Professional CUDA C Programming still show up in nearly every “best CUDA books” thread, and both are cheap used. Their code is where you pay for that. Here is what their samples did on each toolkit:

Book	Published	CUDA 12.9	CUDA 13.4	What breaks
CUDA by Example (Sanders, Kandrot)	2010	7 of 10	6 of 10	Texture references, deviceOverlap, a host-only cuComplex constructor
Professional CUDA C Programming (Wrox)	2014	Stock Makefiles fail, 5 of 7 at sm_75	Stock Makefiles fail, 5 of 7 at sm_75	-arch=sm_20, __shfl without _sync
Learn CUDA Programming (Packt)	2019	5 of 7	5 of 7 (vector add only with SMS overridden)	compute_52, missing helper_functions.h
CUDA Programming (Cook)	2012	No official code found		n/a

Counts are distinct sample programs. CUDA by Example came from NVIDIA’s original 2010 zip. Its texture chapter fails on both toolkits, because the texture reference API was removed in CUDA 12:

chapter07/heat.cu(28): error: texture is not a template

Its streams chapter compiles on 12.9 and breaks on 13.4, because CUDA 13 removed a device property the book reads to check for overlap support:

chapter10/basic_single_stream.cu(40): error: class "cudaDeviceProp" has no member "deviceOverlap"

The Wrox samples, taken from a widely used GitHub mirror because the Wrox download returned HTTP 403, put -arch=sm_20 in every Makefile, which both toolkits reject, so the stock build fails before compiling anything. Passing -arch=sm_75 by hand got five of them through on both toolkits. The two shuffle samples still stop on identifier "__shfl_xor" is undefined, because the old shuffle intrinsics without a mask do not exist for Volta and newer targets. CUDA 12.9 still builds them for sm_60, a target CUDA 13.4 rejects. If you still want to learn from one of these books, plan for these fixes:

1. Change every -arch or SMS value to an architecture your card and toolkit share.
2. Rewrite __shfl calls as __shfl_sync with a full-warp mask.
3. Replace texture references with texture objects.
4. Read asyncEngineCount wherever the code reads deviceOverlap.
5. Copy the Common helpers from NVIDIA’s cuda-samples repository, because the toolkit no longer ships them.

That is a fair amount of porting for a beginner who cannot yet tell a book bug from their own. Learn CUDA Programming is in the table only as a code-rot example: it was out of stock on Amazon, so it is not a pick. The CUDA Toolkit release notes list every removal by version if you hit one we did not.

Is the PMPP 5th edition worth it over the 4th?

Yes, for most buyers, and the price gap is small. On October 28 the 2022 4th edition sold for about $74 with scarce stock, against about $90 for the 5th, so you pay roughly $16 more for the newer book:

• The 5th edition runs 680 pages against 580 in the 4th.
• Its contents add filtering, wavefront algorithms, the multi-GPU API and advanced matrix multiplication.
• Both editions share the same authors and the same pattern-based structure.

Keep the 4th edition if a course you are taking assigns its chapter numbers, or if you already own it and only need the matrix multiplication material, which recent blog posts and the CUDA for LLMs book also cover. Everyone else should buy the 5th.

Which CUDA book should you read first?

Pick by the code you write today, then add a second book when you know where your time goes. Before you open any of them, get a working toolkit and driver: our guide to installing NVIDIA drivers and CUDA on Ubuntu covers it, and nvidia-smi tells you which architecture your card reports. Then follow the path that matches you:

1. New to GPUs and comfortable in C: read Programming Massively Parallel Processors, 5th edition.
2. Writing modern C++: read Motta’s book, then PMPP for the theory.
3. Working in Python: start with Cautaerts and Ghorbanfekr, and pair it with our PyTorch book picks if you train models.
4. Writing LLM kernels: read PMPP first, then CUDA for LLMs when the print edition ships.
5. Tuning training or inference on Blackwell: go straight to Fregly.
6. Running simulations on a cluster: read Robey and Zamora, with Ansorge for the CUDA detail.

Whichever you pick, run nvcc --list-gpu-arch before chapter 1 and compare it with the architecture flags in the book’s build files. That one check would have saved us most of the failures in this test, and C++ readers who want a refresher on the host side can start with our C and C++ book list.