Repository navigation
Add support for blackwell and Cuda 13.3 for the cuda build #3296
Description
Use case
Fix the issue with:
#3289
Implemented the fix for:
Cuda implementation broken
C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>llama-funasr-sensevoice.exe -m sensevoice-small-q8.gguf -a sample.wav --backend cuda
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16310 MiB):
Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 16310 MiB
C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>dir sample.wav
Volume in drive C has no label.
Volume Serial Number is ---
Directory of C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda
07/19/2026 05:15 AM 1,077,326 sample.wav
1 File(s) 1,077,326 bytes
0 Dir(s) 89,731,526,656 bytes free
C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>
C:\Users\USER\Downloads\funasr-llamacpp-windows-x64-cuda>llama-funasr-sensevoice.exe -m sensevoice-small-q8.gguf -a sample.wav
I'll record myself in English, no, you don't want.
[sensevoice] done 1.55s
Proposed solution
Add support for this card and cuda version!
Alternatives or workarounds
None
Expected impact
Improve performance for cuda
Activity
LauraGPT commentedon Jul 22, 2026
Thanks for testing on RTX 5060 Ti / Blackwell. This is a known limitation of the current prebuilt Windows CUDA package: the v0.1.7 CUDA zip targets CUDA architecture 86, while RTX 50 / Blackwell reports compute capability 12.0 (sm_120). The README now calls this out in the llama.cpp / GGUF section.\n\nCurrent workarounds:\n\n1. Use the CPU package for now.\n2. Build the llama.cpp runtime from source with CUDA architecture 120, for example by setting
-DCMAKE_CUDA_ARCHITECTURES=120in the CUDA build.\n\nWe should publish a dedicated Blackwell CUDA artifact once the build matrix is updated and validated, but the existing zip cannot be made compatible at runtime. I am closing this as tracked by the current README/release limitation rather than keeping a duplicate bug open.LauraGPT commentedon Jul 22, 2026
Closing as documented/currently unsupported by the prebuilt sm_86 CUDA artifact. We can reopen or create a release task when the Blackwell CUDA package is ready.
LauraGPT commentedon Aug 29, 2026
重新打开。把“当前预构建包不支持 sm_120”记录到文档并不能完成“支持 RTX 5060 Ti / Blackwell”的请求。
截至
runtime-llamacpp-v0.2.5，Windows CUDA 资产仍固定为架构 86，Blackwell 仍需自行源码构建。因此该 issue 作为明确的预构建资产缺口保持打开，直到发布 sm_120/兼容 Blackwell 的 Windows CUDA 包，并由真实 RTX 50 系用户验证。LauraGPT commentedon Aug 30, 2026
A dedicated Blackwell package is now published, and this issue remains open for your RTX 5060 Ti hardware result.
Please retest with:
Run the same command from the extracted directory:
Please paste the complete output, especially device enumeration, backend initialization, compute status, transcript, and exit code. Also include the NVIDIA driver version.
Precise build boundary: this asset targets sm_120 and was built with CUDA compiler 13.2.51. Its bundled cuBLAS DLLs report 13.3.0 and match NVIDIA official libcublas-windows-x86_64-13.3.0.5; the package includes those DLLs and does not require a separate CUDA Toolkit or Visual C++ redistributable. All ten release jobs and archive/digest audits passed, but that is not evidence of successful RTX 5060 Ti inference. We will keep the issue open until the hardware result is available.