[slaren]

Changes:

• Uses GGML_BACKEND_DL and GGML_CPU_ALL_VARIANTS to build the windows releases to enable dynamic loading of backends
• Uses the clang compiler instead of msvc when possible for better performance
• Creates a single release for the windows x64 CPU build
• Removes the Kompute build
• Removes the outdated arm64-msvc build and cmake toolchain file
• Changes sccache to ccache, since it actually works well enough on windows and supports evict-old-files
• Disables build of test-quantize-stats.cpp with GGML_BACKEND_DL
• Disables outdated AVX-512 test with emulator
• Removes -march=native from llvm cmake toolchain file

Notes:

• I was not able to build the Vulkan release with clang on windows. Fixing this may improve performance of the Vulkan release with partial offloading (cc: @0cc4m @jeffbolznv)

Test run: https://github.com/slaren/llama.cpp/actions/runs/14762791544/job/41447243958
Test release: https://github.com/slaren/llama.cpp/releases/tag/b5235

[jeffbolznv]

Can you provide more details on the clang Vulkan issue and how to reproduce it (or maybe file an issue)? Did you end up just using msvc for Vulkan instead?

[slaren]

Yes, it is still building the Vulkan release with msvc, same as before, but (at least) it also has the multiple CPU variants which should give it better compatibility with different CPUs. When I tried to build Vulkan with clang it failed with this error:

 [84/271] Performing install step for 'vulkan-shaders-gen'
FAILED: ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/Release/vulkan-shaders-gen-install D:/a/llama.cpp/llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/Release/vulkan-shaders-gen-install 
C:\Windows\system32\cmd.exe /C "cd /D D:\a\llama.cpp\llama.cpp\build\ggml\src\ggml-vulkan\vulkan-shaders-gen-prefix\src\vulkan-shaders-gen-build && "C:\Program Files\CMake\bin\cmake.exe" --install . && "C:\Program Files\CMake\bin\cmake.exe" -E touch D:/a/llama.cpp/llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/Release/vulkan-shaders-gen-install"
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:50 (file):
  file INSTALL cannot find
  "D:/a/llama.cpp/llama.cpp/build/bin/Release/vulkan-shaders-gen.exe": No
  error.

Here is the full log: https://github.com/slaren/llama.cpp/actions/runs/14762355462/job/41445824619

[jeffbolznv]

OK, I've heard about this before, I think clang puts the exe in a different folder, we probably need some small change to the cmake file. I'll try to reproduce this soon.

[slaren]

I couldn't reproduce it locally. I suspect that it has something to do with this message while configuring cmake:

 -- Host compiler: C:/mingw64/bin/gcc.exe C:/mingw64/bin/g++.exe

It seems that it thinks that it is cross-compiling and uses a different compiler to build the shader-gen? Not sure what's going on there.

[jeffbolznv]

I tried building locally and while it eventually failed on some curl issue, it did get past the vulkan-shaders-gen part of the build.

Looking at the log again, I noticed this mismatch of Debug vs Release:

[2/2] Linking CXX executable D:\a\llama.cpp\llama.cpp\build\bin\Debug\vulkan-shaders-gen.exe

[84/271] Performing install step for 'vulkan-shaders-gen'
FAILED: ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/Release/vulkan-shaders-gen-install D:/a/llama.cpp/llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/Release/vulkan-shaders-gen-install 
C:\Windows\system32\cmd.exe /C "cd /D D:\a\llama.cpp\llama.cpp\build\ggml\src\ggml-vulkan\vulkan-shaders-gen-prefix\src\vulkan-shaders-gen-build && "C:\Program Files\CMake\bin\cmake.exe" --install . && "C:\Program Files\CMake\bin\cmake.exe" -E touch D:/a/llama.cpp/llama.cpp/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/Release/vulkan-shaders-gen-install"
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:50 (file):
  file INSTALL cannot find
  "D:/a/llama.cpp/llama.cpp/build/bin/Release/vulkan-shaders-gen.exe": No
  error.

Maybe this issue is specific to the ninja multi-config?

[slaren]

One possible issue with this change that I didn't realize at first is that the examples that are not compatible with GGML_BACKEND_DL will no longer be included in the binary distribution. This includes these examples:

• convert-llama2c-to-ggml
• cvector-generator
• export-lora
• llava
• rpc

The most impactful of these are likely to be the llava and the rpc server. cc @ngxson @rgerganov

Fixing this wouldn't be complicated. Essentially:

• Add a call to ggml_backend_load_all on startup to load the backends
• Use the backend registry instead of the backend-specific functions

[rgerganov]

The most impactful of these are likely to be the llava and the rpc server. cc @ngxson @rgerganov

I am traveling and won't be able to address this in the next few days, sorry. You can exclude rpc-server as stop-gap solution