opencl: keep half-precision matrix-vector products on the GEMM path - #3727
Draft
melonakos wants to merge 2 commits into
Draft
opencl: keep half-precision matrix-vector products on the GEMM path#3727melonakos wants to merge 2 commits into
melonakos wants to merge 2 commits into
Conversation
CLBlast's f16 GEMV returns wrong values (CNugteren/CLBlast#561) while its GEMM does not, so route half inputs through GEMM even when the right-hand side is a single column. This is what produced NaNs in fp16 strided convolutions on Intel GPUs. Adds a random half matrix-vector test checked against f32. Fixes #3674
melonakos
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On OpenCL a fp16 matrix times a single column goes to CLBlast's half GEMV. With the same fp16 data the GEMM path is correct and the GEMV path is not, on an Intel Arc B580 (driver 32.0.101.8991) and a UHD 770 (32.0.101.7088), with the pinned CLBlast and with 1.7.0; that GEMV call is the one #3674's strided fp16 convolutions make when the unwrapped input is multiplied by the single-column filter. CLBlast#561 reports wrong half GEMV results on another vendor and may be related, but it is unresolved and its symptom differs. This keeps half inputs on the GEMM path and adds a random half matrix-vector test compared against f32.
Works around #3674 on the two Intel GPUs above: the ConvolveStrided cases that failed there now pass on repeat, and the blas and convolve suites pass. #3674 also reports the failure on NVIDIA and on the OpenCL CPU device, where this is not verified, so it does not close the issue.