Skip to content

Linux native packages do not register /opt/rocm/lib with the dynamic loader #6716

Open

Description

@liminfei-amd

Problem Description

TheRock Linux native packages create the intended compatibility path:

/opt/rocm/lib -> /etc/alternatives/rocm-lib -> /opt/rocm/core-X.Y/lib

but do not install an entry under /etc/ld.so.conf.d for that path. As a result, an external native program can link against ROCm successfully and then fail at startup because its DT_NEEDED libraries are absent from the loader cache.

This reproduces in the official rocm/dev-ubuntu-24.04:7.14.0-full image (digest sha256:439edaa8f0c4be4a3728e528f87b8a2ea1f051f34cf10b27caa4bd94f562eda7) without GPU access:

cat >/tmp/probe.cpp <<'EOF'
#include <hip/hip_runtime_api.h>
int main() {
    int version = 0;
    return hipRuntimeGetVersion(&version) == hipSuccess ? 0 : 1;
}
EOF

hipcc /tmp/probe.cpp -o /tmp/probe
env -u LD_LIBRARY_PATH /tmp/probe

Result:

error while loading shared libraries: libamdhip64.so.7: cannot open shared object file
exit code: 127

ldd also reports libamdhip64.so.7 => not found. Bare lookup similarly fails for libhipblas.so.3 and librocblas.so.5.

Adding /opt/rocm/lib to /etc/ld.so.conf.d and running ldconfig changes the same binary to exit 0. It also fixes the user-reported case in ggml-org/llama.cpp#25807: an unmodified llama.cpp build that otherwise exits 127 on libhipblas.so.3.

The same failure was reproduced with the stable ROCm 7.11 package path from the commit that introduced package-based runtime images, so this is not specific to llama.cpp or newly introduced by 7.14.

Expected Behavior

Programs linked against a system-level ROCm DEB/RPM installation should resolve the active ROCm libraries without requiring LD_LIBRARY_PATH or application-specific absolute RUNPATHs.

The stable loader entry should follow the active update-alternatives selection and refresh correctly across install, upgrade, version switching, removal, and purge/erase.

Scope

This report is limited to Linux native DEB/RPM packages. It does not propose global loader configuration for Python virtual environments, tarballs, arbitrary prefixes, or Windows DLL packages.

Environment

  • Operating system: Ubuntu 24.04 container
  • ROCm version: 7.14
  • Installation method: TheRock Linux native packages
  • GPU: not required to reproduce

Activity

  1. Samiii777 commented on Aug 12, 2026

    @Samiii777
    Contributor

    Confirming this on bare-metal Ubuntu 26.04 with the repo.radeon.com/rocmradeon/apt/26.13 packages (ROCm 7.14.0~pre3, gfx1100), and adding the root cause, which I think makes this a one-line packaging fix.

    The conf file is already built and shipped — it is just never activated.

    amdrocm-opencl7.14 installs a correct ld.so.conf.d fragment inside the ROCm tree:

    $ dpkg -L amdrocm-opencl7.14 | grep ld.so
    /opt/rocm/core-7.14/etc/ld.so.conf.d
    /opt/rocm/core-7.14/etc/ld.so.conf.d/10-rocm-opencl.conf
    
    $ cat /opt/rocm/core-7.14/etc/ld.so.conf.d/10-rocm-opencl.conf
    # Default value.
    # It will be replaced by package script, such as postinst, during installation.
    /opt/rocm/lib
    

    The file's own comment states a maintainer script is supposed to consume it, but no such script exists:

    $ ls /var/lib/dpkg/info/amdrocm-opencl7.14.*
    amdrocm-opencl7.14.list  amdrocm-opencl7.14.md5sums
    amdrocm-opencl7.14.shlibs  amdrocm-opencl7.14.triggers      # no .postinst
    
    $ grep -l 'ld.so.conf\|ldconfig' /var/lib/dpkg/info/amdrocm*.postinst
    (no matches — 26 amdrocm postinst scripts exist, none reference ldconfig)
    

    So the payload is correct and the wiring is missing: nothing ever copies the fragment to /etc/ld.so.conf.d/ or runs ldconfig.

    Reproduces exactly as described:

    $ hipcc --offload-arch=gfx1100 probe.hip -o probe
    $ env -u LD_LIBRARY_PATH ./probe
    ./probe: error while loading shared libraries: libamdhip64.so.7: cannot open shared object file: No such file or directory
    
    $ ldconfig -p | grep -c libamdhip64
    0
    

    Worth noting the binaries also carry no RPATH/RUNPATH, so the loader cache is the only thing that can resolve them:

    $ readelf -d ./probe | grep -E 'RPATH|RUNPATH'
    (nothing)
    

    Fix confirmed — installing the fragment ROCm already ships is sufficient:

    $ echo /opt/rocm/lib | sudo tee /etc/ld.so.conf.d/10-rocm.conf
    $ sudo ldconfig
    $ env -u LD_LIBRARY_PATH ./probe && echo OK
    OK
    

    There is already a working reference implementation in the sibling AMD stack — amdgpu-core.postinst does exactly this for /opt/amdgpu:

    conf=/etc/ld.so.conf.d/20-amdgpu.conf
    if [ "$1" = "configure" ]; then
      echo "$prefix/$lib" >$conf
      ...
    fi

    which is why /etc/ld.so.conf.d/20-amdgpu.conf is present on the same machine while no ROCm equivalent is.

    One packaging caveat for whoever picks this up: with the versioned /opt/rocm/core-X.Y layout plus /etc/alternatives, the fragment should point at the stable /opt/rocm/lib symlink (as the shipped file already does) rather than a version-pinned path, so it survives alternative switches; and the corresponding postrm should remove it so parallel-installed versions don't leave a dangling entry.

  2. bernardogv commented on Oct 5, 2026

    @bernardogv

    Still reproduces on ROCm 10.1.0-3 native debs, Ubuntu 26.04.1 LTS (kernel 7.0.0-38-generic), gfx1151 (Ryzen AI Max+ 395 / Radeon 8060S).

    After apt install amdrocm-core-sdk10.1-gfx1151 from stable.repo.amd.com/rocm/core/packages/ubuntu2604, ldconfig -p | grep libhsa-runtime64 is empty. The libraries are in /opt/rocm/core-10.1/lib, and nothing under /etc/ld.so.conf.d/ points there.

    Workaround:

    echo /opt/rocm/core-10.1/lib | sudo tee /etc/ld.so.conf.d/rocm.conf && sudo ldconfig
    

    After that, rocminfo reports gfx1151, and a hipcc-built kernel (--offload-arch=gfx1151) runs correctly as a non-root user in render/video.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    packaging: linuxIssues relating to Linux packaging

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions