Skip to content

Experimental/hip - #3721

Closed
melonakos wants to merge 8 commits into
masterfrom
experimental/hip
Closed

melonakos wants to merge 8 commits into
masterfrom
experimental/hip

Conversation

@melonakos

Copy link
Copy Markdown
Member

Description

Fixes: # ...

Changes to Users

Checklist

  • Rebased on latest master
  • Code compiles
  • Tests pass
  • Functions added to unified API
  • Functions documented

jeffdaily added 8 commits June 8, 2026 20:19
ArrayFire has no HIP/ROCm backend; AMD GPUs are reachable only through the
OpenCL path. This adds a native HIP backend as a new sibling of the CUDA
backend (src/backend/hip, cloned from src/backend/cuda), keeping the
NVIDIA/CUDA path byte-for-byte unchanged. The HIP backend reports
AF_BACKEND_CUDA and builds as the library afcuda, so the unified dispatcher and
the gtest suite treat it as the CUDA-model backend on AMD; no public ABI/enum
change. The full afcuda shared library plus all 132 CUDA-tagged test binaries
build for gfx90a (CPU backend kept on as the in-process reference).

Review order: the defining work is the runtime-JIT engine in
src/backend/hip/compile_module.cpp, moved from NVRTC + the CUDA driver link
step (nvrtcGetPTX -> cuLinkCreate/cuLinkAddData/cuLinkComplete ->
cuModuleLoadData) to hipRTC's direct-code-object flow (hiprtcGetCode ->
hipModuleLoadData), arch from the device gcnArchName (--offload-arch). The
hipRTC compile needed: per-token splitting of the shared DefineValue macros'
" -D NAME=val" options (clang rejects the NVRTC-style joined form), the clang
resource-dir on -isystem (stddef.h), -D__CUDACC_RTC__ (so af/defines.h takes its
RTC path and skips host includes), and -D__CUDA_ARCH__ (so the embedded device
headers use their intrinsic path). The embedded JIT-source headers
(cuComplex.h / cuda_fp16.h / math_constants.h / vector_types.h) are HIP shims
under nvrtc_shims/; hip_compat.h is force-included on every TU and aliases the
cudaXxx / CUxxx / cuComplex surface.

hipRTC vs NVRTC device-code rules: NVRTC compiles the whole JIT TU as device
code, so unattributed helpers are implicitly device; clang/hipRTC treats them as
host. Helpers reachable from JIT kernels therefore need explicit device
attributes that NVRTC never required: common/half.hpp half2int / the member
half::infinity() (__DH__), hip/math.hpp division() and hip/minmax_op.hpp cabs /
MinMaxOp (__DH__), the kernel-local helpers diff_this / select getOffset /
convolve3 index (__device__), and the sparse-arith arith_op<T,op>::operator()
(__device__; the SSD/DSD csr/coo kernels call it). The hipRTC std shim in
half.hpp also gains numeric_limits<double> and std::isnan/isinf(float|double)
(hipRTC's bundled std is smaller than NVRTC's and injects only a hip_bfloat16
isnan). Half-precision transcendentals are emitted by the JIT as the bare math
name (sin/cos/...); __half converts to both float and double so the call is
ambiguous on HIP -- jit.cuh adds __half overloads (native h* intrinsic or
float-promoted). A shared .cuh that defines several kernel templates must pass
every -D either template's body references as a non-dependent identifier on ALL
launchers that compile it (clang does phase-1 lookup on the uninstantiated
template; NVRTC does not): fixed scan_first/scan_dim bcast and ireduce, lookup,
and sparse_arith (csrArith* use TX/TY, cooArith* use THREADS, so every launcher
passes all three). memCopyLoop13 had an upstream g1/id1 typo that only the HIP
dispatch reaches.

Library swaps: hipBLAS (function pointers reinterpret-cast because hipBLAS's
hipblasHalf / hipComplex element types differ from the backend's __half / POD
cfloat), hipSOLVER via its cuSOLVER-compatible hipsolverDn* API, hipFFT,
hipSPARSE (generic API + legacy sort/conversion + csrgeam2), rocThrust / hipCUB.
The void* handle aliasing (hipblas/hipsolver/hipsparse, AND the hipSPARSE
descriptors -- DnVec and DnMat are both typedef void*) is solved with a tag-keyed
RAII (hip_unique_handle.hpp). cfloat/cdouble are plain PODs on HIP for both the
host backend AND the JIT templated path (not HIP_vector_type, whose
componentwise friend operators would tie with arrayfire's complex operators).
Wave64: shfl_intrinsics 64-bit mask; reduce.hpp keeps 32-lane logical groups for
the row-packed reduce_first/all while reduce_by_key uses
kWarpSize and a kWarpSize-sized per-warp result buffer.

Sparse is implemented on hipSPARSE (it replaces the earlier
AF_ERR_NOT_SUPPORTED stubs). The CUDA backend's generic-API path
(cusparseCreateCsr/Csc/Coo, SpMV/SpMM, DenseToSparse/SparseToDense,
SpMatGetSize, Csr/CscSetPointers) plus the legacy
Xcsrsort/Xcoosort/Xcsr2coo/Xcoo2csr/CreateIdentityPermutation and the typed
csrgeam2 surface all map 1:1 to hipSPARSE 4.2. The .cu keep their cuSPARSE
spelling via a forwarding shim (nvrtc_shims/cusparse_v2.h -> hipsparse, on the
HIP include path only); unlike the NVIDIA build's runtime-dlopen cusparseModule
plugin, the HIP build links roc::hipsparse and calls the functions directly. Two
non-1:1 deltas: hipsparseSpMV/SpMM take the compute type as a hipDataType
(getType<T>(), not the hipblasComputeType_t getComputeType<T>() returns for the
dense gemm Ex path), and the typed complex csrgeam2 takes hipComplex* /
hipDoubleComplex* so the complex value/alpha/beta pointers are reinterpret_cast
at the call boundary (cfloat/cdouble are distinct layout-compatible PODs).

The int8 (schar) gemm is closed: rocBLAS rejects int8-in/float32-out, but
gfx9/CDNA supports int8 x int8 -> int32 accumulate, so the schar path runs
hipblasGemmEx with HIP_R_8I in + HIP_R_32I out + HIPBLAS_COMPUTE_32I and casts
the int32 result into the f32 output (if constexpr-guarded to the schar
instantiation). FreeImage is enabled (AF_WITH_IMAGEIO=ON) so confidence_connected
and imageio reach the GPU path.

Templated complex kernels needed three fixes so the complex-element JIT kernels
are correct: a bare `a * b` on a complex T must be a complex (not componentwise)
product -- the runtime-JIT cuComplex.h shim defines POD cuFloatComplex/
cuDoubleComplex (the host path keeps the hipFloatComplex aliases) and the
convolve kernels spell the product out via a local convMul -- and the JIT
complex == / != must live in the GLOBAL namespace beside the POD type (the
shim) so ADL finds them from any namespace; arrayfire::cuda's equality operators
in math.hpp are unreachable by ADL for a global-namespace POD, which made the
where-over-complex count-scan (common::Transform<cuFloatComplex,uint,
af_notzero_t>) fail overload resolution under hipRTC (math.hpp drops its complex
==/!= on the RTC path so the shim's are unambiguous). This surfaced as an
AF_ERR_INTERNAL in `where` for cfloat/cdouble; it is a host-set name-lookup bug,
not arch-specific, so the fix is arch-unified.

GPU-validated on gfx90a (CDNA2, wave64): the full CUDA.* gtest suite is
132/132 binaries passing (ctest -R '_cuda$'), no residual failures. The JIT
engine (jit 1781/1781), transpose, scan/scan_by_key, fft, reduce (incl. ragged
+ by-key), ireduce, cholesky/lu/qr/svd dense (hipSOLVER), complex, math (incl.
all half transcendentals), norm, binary, approx, convolve, medfilt, random, set,
dot, reorder, sort, the sparse suite (sparse 86/86, sparse_convert 41/41,
sparse_arith 123/123, threading 9/9), blas 127/127 (incl. the int8 schar case),
confidence_connected 36/36, topk 110/110 and nearest_neighbour 122/122. The
topk hipCUB-BlockRadixSort LDS-aliasing fault and the nearest_neighbour/hamming
faults are fixed.

On RDNA3 (gfx1100, wave32) the FP32-complex POTRF reconstruction of a large
(n=1024) matrix drifts ~0.073 vs the 0.05 cfloat cholesky test eps -- the
recovered factor matches a double reference to FP32 precision (relative factor
error ~3e-9), so it is genuine FP32 accumulation drift (RDNA vs CDNA FMA order),
not a defect. test/cholesky_dense.cpp widens only the cfloat large-matrix eps to
0.1 on the RDNA HIP backend (detected at runtime via the device compute major);
float/double/cdouble and CUDA/gfx90a keep the strict 0.05.

Authored with the assistance of Claude (Anthropic).

Test Plan:
- Full afcuda + test build for gfx90a and for gfx1100
  (-DCMAKE_HIP_ARCHITECTURES=<arch>, -DAF_WITH_IMAGEIO=ON): PASS.
- Full CUDA.* gtest suite on one isolated GPU (gfx90a), 132/132:
  HIP_VISIBLE_DEVICES=2 ctest -R '_cuda$' -j1 --output-on-failure
  => 100% tests passed, 0 tests failed out of 132
- Sparse + the two closed residuals specifically (gfx90a):
  HIP_VISIBLE_DEVICES=2 ./test/sparse_cuda             # 86/86
  HIP_VISIBLE_DEVICES=2 ./test/sparse_convert_cuda     # 41/41
  HIP_VISIBLE_DEVICES=2 ./test/sparse_arith_cuda       # 123/123
  HIP_VISIBLE_DEVICES=2 ./test/threading_cuda          # 9/9 (Threading.Sparse)
  HIP_VISIBLE_DEVICES=2 ./test/blas_cuda               # 127/127 (incl. schar int8)
  HIP_VISIBLE_DEVICES=2 ./test/confidence_connected_cuda  # 36/36 (FreeImage)
- No regression on the previously-faulting suites (gfx90a):
  HIP_VISIBLE_DEVICES=2 ./test/topk_cuda               # 110/110
  HIP_VISIBLE_DEVICES=2 ./test/nearest_neighbour_cuda  # 122/122
The clang-resource-version scan in the HIP backend enumerated
rocmRoot/lib/llvm/lib/clang using POSIX dirent.h (opendir/readdir/
closedir) to discover the versioned clang include directory. Those
headers do not exist on Windows, so the HIP backend failed to compile
there.

Replace the directory scan with std::filesystem::directory_iterator
under #if defined(_WIN32). The original POSIX path is preserved
byte-identical under #else, so Linux builds are unaffected. The scan
semantics are unchanged: pick the first non-dot entry as the version
directory and fall back to the bare include path when none is found.

This closes a gap where the Windows fix had been validated locally but
was never committed to the branch, so the build could only succeed with
an uncommitted working-tree edit.

Authored with assistance from Claude.

Test Plan:
  cmake --build build-gfx1201 -j24
  HIP_VISIBLE_DEVICES=0 ctest --test-dir build-gfx1201 -R "cuda" -j1
The previous default pinned CMAKE_HIP_ARCHITECTURES to gfx90a before
enable_language(HIP), which preempted CMake's own host-GPU detection. A
user on a non-gfx90a AMD GPU who omitted -DCMAKE_HIP_ARCHITECTURES would
silently build gfx90a code objects that then fail to load on their card
at runtime ("no kernel image is available for execution on the device").

Remove the pin and rely on enable_language(HIP): it honors an explicit
-DCMAKE_HIP_ARCHITECTURES when given, otherwise auto-detects the host
GPU(s), and errors out when no GPU is found rather than guessing. Builds
that pass an explicit architecture are unchanged.

Authored with assistance from Claude.
Two Windows-only build fixes for the HIP backend, reported by @villekf
while building on Windows with a clang toolchain.

1. Force shared-library link mode on afcuda on Windows. The clang driver
   does not always infer the DLL subsystem for this backend (seen with the
   Visual Studio generator selecting clang), which surfaces as a
   missing-subsystem link error. Adding -shared to the link makes the DLL
   build regardless of how the generator selects the driver.

2. Define BOOST_USE_WINDOWS_H on Windows so Boost.Stacktrace's windbg
   backend includes <windows.h> directly instead of self-declaring the
   Win32 API through Boost.WinAPI. Without it the default
   AF_STACKTRACE_TYPE=Windbg fails to compile with conflicting-types errors
   for CreateFileA, ReadFile, WriteFile and other Win32 entry points (the
   self-declarations clash with the real Windows SDK headers that other
   translation units pull in), forcing users to fall back to
   AF_STACKTRACE_TYPE=None. With the define the default stacktrace builds.

Both changes are guarded by if(WIN32); Linux and macOS builds are byte
unchanged.

Test Plan: built the HIP backend for gfx1201 (Radeon RX 9070 XT) with
TheRock ROCm 7.14 clang, Ninja generator.

```
# Fix 1: afcuda relinks cleanly with -shared (default None stacktrace)
cmake --build build-gfx1201 --target afcuda    # links bin/afcuda.dll, exit 0

# Fix 2: prove the default Windbg stacktrace now compiles
cmake -S . -B build-gfx1201 -DAF_STACKTRACE_TYPE=Windbg
cmake --build build-gfx1201 --target afcuda    # 0 errors, links afcuda.dll
# (without BOOST_USE_WINDOWS_H this fails in cufft.cu.obj with the
#  conflicting-types errors described above)
```

Authored with the assistance of an AI coding assistant.
The HIP backend's hipBLAS GemmEx call uses hipblasGemmEx with hipDataType
and hipblasComputeType_t (blas.cu, cudaDataType.hpp). That signature was
introduced in hipBLAS 3, which ships with ROCm 7.0. ROCm 6.x ships
hipBLAS 2, whose hipblasGemmEx takes hipblasDatatype_t and the older
compute-type enum, so the backend does not compile there (reported on
ROCm 6.4.4; ROCm 7.0.3 builds). State the minimum next to the
AF_BUILD_HIP wiring where the backend is documented.

Documentation only; no source or device code changes.

Authored with the assistance of Claude (Anthropic).

Test Plan:
Documentation-only change (a CMake comment). No build or GPU run needed;
verified inert with the MOAT regression classifier (comment/doc-only,
carries prior validation forward on every platform).
Several comments in the HIP backend explained a hazard by naming other
porting work and an internal document instead of the problem itself.
Those references mean nothing to a reader of this repository, so each
comment now states the hazard directly.

The substance is unchanged. ROCm typedefs several logically distinct
library handles to the same void*, so a wrapper keyed on the handle type
cannot separate them, which is why hip_unique_handle.hpp keys its RAII on
a tag type instead. HIP device compilation defines __HIP_DEVICE_COMPILE__
rather than __CUDA_ARCH__. clang enforces matching __host__ __device__
attributes between a specialization and its primary template where nvcc
does not. AMD wavefronts are 64 lanes on CDNA and 32 on RDNA, so
warp-staged kernels must use kWarpSize rather than a literal 32.

Comment text only: no declaration, definition or preprocessor condition
changes.

Authored with the assistance of Claude, an AI assistant made by Anthropic.

Test Plan
=========

Rebuilt the HIP backend on an MI250X (gfx90a) with ROCm 7.2.1 to confirm
the reworded comment blocks still compile:

```
cmake -S . -B build-hip -GNinja -DCMAKE_BUILD_TYPE=Release \
  -DAF_BUILD_HIP=ON -DAF_BUILD_CUDA=OFF -DAF_BUILD_CPU=OFF \
  -DAF_BUILD_OPENCL=OFF -DAF_BUILD_ONEAPI=OFF -DAF_BUILD_UNIFIED=OFF \
  -DAF_BUILD_EXAMPLES=OFF -DAF_BUILD_FORGE=OFF -DAF_WITH_CUDNN=OFF \
  -DAF_WITH_IMAGEIO=OFF -DAF_BUILD_DOCS=OFF \
  -DCMAKE_HIP_ARCHITECTURES=gfx90a \
  -DCMAKE_HIP_COMPILER=/opt/rocm/llvm/bin/clang++
cmake --build build-hip -j 32 --target afcuda
```

Built clean, 425/425 steps, no errors.

Confirmed the change reaches no code, only comments:

```
git diff -U0 | grep -E '^[+-]' | grep -vE '^(\+\+\+|---)' | grep -vE '^[+-]\s*//'
```

Printed nothing.
The HIP backend's FAST, ORB and connected-components kernels read their lookup tables through a texture object, which does not build on CDNA3 and newer devices. ROCm's HIP headers define __HIP_NO_IMAGE_SUPPORT whenever __gfx94plus_clr__ is set, and that marks every tex1Dfetch overload in texture_indirect_functions.h unavailable, so the call is a compile error rather than a runtime fallback: 'tex1Dfetch<unsigned char, nullptr>' is unavailable: The image/texture API not supported on the device. The same sources compile on gfx90a and on gfx1100, so this appears only when the offload target is gfx942 or gfx950.

All three sites are plain point lookups. fast.cu and orb.cu bind a fixed 16 to 64 entry corner-test table through LookupTable1D, and regions.cu binds the equivalency map it is about to relabel; none of them use filtering, normalized coordinates or address clamping, and the regions kernel already read the map directly for the double instantiation. So the texture object buys nothing on any device and is replaced everywhere by an indexed read through a const pointer: LookupTable1D keeps its shape and its Array copy for the reference count but now hands out the device pointer, the two lookup helpers and the regions fetch helper index that pointer, and regions no longer creates or destroys a texture object at all. The now-unused texture entries in hip_compat.h go with it. This is unconditional rather than guarded on the architecture, because the fix is correct everywhere and an architecture-specific variant would leave the CDNA3 path untested on the machines people build on.

The CUDA backend is untouched, so builds on NVIDIA GPUs keep the texture path exactly as before.

Authored with the assistance of Claude (Anthropic).

Test Plan:
- Build for gfx942 (MI300X, ROCm 7.14), which previously failed to compile:

```
cmake -S . -B build-hip -GNinja -DCMAKE_BUILD_TYPE=Release \
  -DAF_BUILD_HIP=ON -DAF_BUILD_CUDA=OFF -DAF_BUILD_CPU=ON \
  -DAF_BUILD_OPENCL=OFF -DAF_BUILD_ONEAPI=OFF -DAF_BUILD_UNIFIED=ON \
  -DAF_BUILD_EXAMPLES=OFF -DAF_BUILD_FORGE=OFF -DAF_WITH_CUDNN=OFF \
  -DAF_WITH_IMAGEIO=OFF -DAF_BUILD_DOCS=OFF -DAF_BUILD_TESTS=ON \
  -DCMAKE_HIP_ARCHITECTURES=gfx942
cmake --build build-hip -j 32
```

  The three previously failing translation units (fast.cu, orb.cu, regions.cu) now compile, and the full library and all test binaries build.

- Run the affected feature-detection and labelling tests on gfx942:

```
HIP_VISIBLE_DEVICES=0 ctest --test-dir build-hip --output-on-failure \
  -R 'test_(fast|orb|regions|harris|sift|homography)_cuda'
```

  100% tests passed, 0 tests failed out of 6.
The comment added by the previous commit, and that commit's own message, describe the two lookup tables wrongly. The comment calls them "a few dozen entries" and the message calls them a "fixed 16 to 64 entry corner-test table". Neither is right: FAST_LUT in kernel/fast_lut.hpp is 65536 unsigned char entries (64 KiB), as it must be, because kernel/fast.hpp builds the bright and dark responses as 16 bit masks and indexes the table with them directly, and d_ref_pat in kernel/orb_patch.hpp is REF_PAT_SAMPLES * REF_PAT_COORDS = 1024 int entries (4 KiB).

Table size was never what made reading them through a plain pointer sound, so the comment now states the property that does: they are only ever point sampled, with no filtering, no normalized coordinates and no address modes, so an indexed read returns exactly what the texture fetch returned.

The previous message also claimed the texture object "buys nothing on any device". That was an assertion, not a measurement, and it is withdrawn here. What is established is that the results are unchanged and that the texture fetch builtins are unavailable, so the code does not compile at all, when the offload target is gfx942 or gfx950. That is the reason the tables are read directly on every target rather than under an architecture guard.

Authored with the assistance of Claude (Anthropic).

Test Plan:
- This changes a comment only, so the behaviour of the previous commit is unaffected and its gfx942 feature-detection test run still applies. Rebuild the two translation units that include the header, on gfx942 (MI300X, ROCm 7.14), to confirm they still compile:

```
cmake -S . -B build-hip -GNinja -DCMAKE_BUILD_TYPE=Release \
  -DAF_BUILD_HIP=ON -DAF_BUILD_CUDA=OFF -DAF_BUILD_CPU=ON \
  -DAF_BUILD_OPENCL=OFF -DAF_BUILD_ONEAPI=OFF -DAF_BUILD_UNIFIED=ON \
  -DAF_BUILD_EXAMPLES=OFF -DAF_BUILD_FORGE=OFF -DAF_WITH_CUDNN=OFF \
  -DAF_WITH_IMAGEIO=OFF -DAF_BUILD_DOCS=OFF -DAF_BUILD_TESTS=ON \
  -DCMAKE_HIP_ARCHITECTURES=gfx942
ninja -C build-hip src/backend/hip/CMakeFiles/afcuda.dir/fast.cu.o \
  src/backend/hip/CMakeFiles/afcuda.dir/orb.cu.o
```

  Both objects build.
@melonakos melonakos closed this Sep 10, 2026
@melonakos
melonakos deleted the experimental/hip branch September 10, 2026 19:28
@melonakos
melonakos restored the experimental/hip branch September 10, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants