PyTorch AMD Build Scripts #6625

Jorghi12 · 2018-04-16T12:37:21Z

The PyTorch script files necessary for building on AMD GPUs.

ezyang · 2018-04-16T17:35:12Z

Hi @Jorghi12, does this mean we are ready to setup AMD builders on CI?

Also, directory structure bikeshedding: can we put this inside tools, e.g., as tools/amd?

amd_build/build_pytorch_amd.py

+import subprocess
+import os
+
+cwd = os.getcwd()


amd_build/build_pytorch_amd.py

+
+cwd = os.getcwd()
+proj_dir = os.path.dirname(cwd)
+out_dir = os.path.join(os.path.dirname(proj_dir), "pytorch_amd")


amd_build/build_pytorch_amd.py

+shutil.copy(os.path.join(cwd, "hip_files/THC/generic/THCTensorRandom.cu.hip"), os.path.join(out_dir, "aten/src/THC/generic/THCTensorRandom.cu"))
+
+# Move to avoid HCC bug.
+shutil.move(os.path.join(out_dir, "aten/src/ATen/native/cudnn/Conv.cpp"), os.path.join(out_dir, "aten/src/ATen/native/cudnn/ConvCuDNN.cpp"))


amd_build/build_pytorch_amd.py

+
+# Execute the Hipify Script.
+subprocess.Popen(
+    ["/opt/rocm/bin/hipify-python.py",


amd_build/hip_files/ATen/CMakeLists.txt.hip

@@ -0,0 +1,454 @@
+CMAKE_MINIMUM_REQUIRED(VERSION 2.8)


amd_build/hip_files/THC/THCApply.cuh.hip

@@ -0,0 +1,878 @@
+#ifndef THC_APPLY_INC
+#define THC_APPLY_INC


ezyang · 2018-04-16T17:43:11Z

Looking at the patch that the build script applies to master, it seems sufficiently big that the next time someone merges a change to anything in THC, it will immediately stop working.

In general, I am wondering, would it make more sense to maintain this as a branch, until we can mainline the source code changes so that they work for AMD as well as CUDA? (I was not privy to any of the original discussions, so I apologize if this is something you've already looked at.)

Use the actual path for the file instead of the current working directory, which depends on where the script is invoked.

…iles with the same name results in a linking error in the HCC compiler used for ROCm/AMD.

…g up relevant hip paths.

…es while building.

Replacing "WITH_CUDA" with "NOT NO_CUDA" after the rebase.

apaszke

LGTM. One minor nit. Should be ready to merge once @ezyang approves the CMake part.

setup.py

    DL = Extension("torch._dl",
                   sources=["torch/csrc/dl.c"],
                   language='c',
+                   extra_link_args=[]


yf225 · 2018-05-15T01:02:13Z

@pytorchbot retest this please

Jorghi12 · 2018-05-16T01:30:43Z

The build for pr/pytorch-linux-xenial-py3-clang5-asan was hanging for 3 hours previously so I restarted the build on Jenkins.

Finished: success https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-linux-xenial-py3-clang5-asan-build/4652/console

…/pytorch#6625) pytorch/pytorch@cd86d4c

* PyTorch AMD Build Script. * Python invocation for hipify * Adding individual hip fles. * Updating CWD Use the actual path for the file instead of the current working directory, which depends on where the script is invoked. * Updating folder path for amd_build * Removing previous amd_build directory * Updated setup.py to support WITH_ROCM * Renaming the files for CuDNN BatchNorm & Conv since having two .cpp files with the same name results in a linking error in the HCC compiler used for ROCm/AMD. * Removing old BatchNorm & Conv files since they've been renamed. * Updating build path to handle ROCM * Cleaned up the build path and created a FindHIP cmake file for setting up relevant hip paths. * Seperated the individual patch files to make it easier to detect issues while building. * Removed CMakeLists hip files and fixed directory structure * Adding build pytorch amd script * Merged setup patch into PyTorch setup.py & cleaned a few issues * Added information on where to download the hipify-python script. * Resolved linting issues inside of build_pytorch_amd.py * Removing many unnecessary patch files. Removing unnecessary .hip files. Fixing up the build process. * Refactored the PR for supporting HIP * Minimizing the number of changes inside individual patches. * Cleaned up patch files. * Removed patch files. * Updating patches * Removing HIP change from file. * Cleaned up patches * Added AVX/SSE avoidance due to bug with ROCms stack. Just temporary for now. * Removing the other HIP file * Removed patch file + merged ROCm into Aten/test * Removed ATen tests patch file and updated disbale_features yaml to remove headers that don't exist on the HIP stack. * Reduced the number of patches down to 14 after Edward's suggestions. * Transferred deletion of certain functions from patch to yaml file. * Set default Thrust path * Fixed aten files so we now use the templated pow/abs instead of std:: directly. * Removed error from aten/src/THCUNN/Abs.cu * Updated the locations of the cmake build files. Moved THCTensorRandom from a hip to a patch file. Added executable/library commands that can successfully handle either CUDA or HIP. * Removed hip extraction from the build script and removed the old hip file. * Replaced MACRO with function in upper level cmake. * Added empty ELSE() block to prevent the loading of a command without CUDA or HIP. Also added IF guards around torch_cuda_based_add_executable in Aten tests. * Updated aten tests. * Removed the hip include from the ATen header. * Can't throw exceptions on C++ AMP, using abort * Missing IF guards for cuda/hip executables in aten tests. * Removed a series of patch files. * Added template keyword to help out the HCC compiler. * Rebased the specific files displayed in the PR * Fixing typo. * Change flag from "WITH_CUDA" to "NOT NO_CUDA" Replacing "WITH_CUDA" with "NOT NO_CUDA" after the rebase. * Fix LoadHIP path * Updating build files after rebasing. * Reorganization after cpu/gpu separation. * Removed HIPCC from setup.py & removed -shared extra linking args. * Updated CMake / Setup build to correctly link when under ROCm stack. * Removed the unnecessary argument from Extension constructor. * Adding another test to be included with ROCm building. * Updated the setup_helpers scripts in order to get around linter error * Fix syntax issue * Solving lint issue: line too long

Jorghi12 added 3 commits April 16, 2018 05:31

PyTorch AMD Build Script.

678e39a

Python invocation for hipify

c52848e

Adding individual hip fles.

62a58c2

ezyang reviewed Apr 16, 2018

View reviewed changes

amd_build/build_pytorch_amd.py Outdated

import subprocess

import os

cwd = os.getcwd()

This comment was marked as off-topic.

Sign in to view

ezyang reviewed Apr 16, 2018

View reviewed changes

amd_build/build_pytorch_amd.py Outdated

cwd = os.getcwd()

proj_dir = os.path.dirname(cwd)

out_dir = os.path.join(os.path.dirname(proj_dir), "pytorch_amd")

This comment was marked as off-topic.

Sign in to view

ezyang reviewed Apr 16, 2018

View reviewed changes

amd_build/hip_files/ATen/CMakeLists.txt.hip Outdated

@@ -0,0 +1,454 @@

CMAKE_MINIMUM_REQUIRED(VERSION 2.8)

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

ezyang reviewed Apr 16, 2018

View reviewed changes

amd_build/hip_files/THC/THCApply.cuh.hip Outdated

@@ -0,0 +1,878 @@

#ifndef THC_APPLY_INC

#define THC_APPLY_INC

This comment was marked as off-topic.

Sign in to view

Jorghi12 and others added 11 commits April 16, 2018 15:05

Updating CWD

20cd167

Use the actual path for the file instead of the current working directory, which depends on where the script is invoked.

Updating folder path for amd_build

b4a6900

Removing previous amd_build directory

3b92636

Updated setup.py to support WITH_ROCM

8bb473c

Renaming the files for CuDNN BatchNorm & Conv since having two .cpp f…

0b3dbc8

…iles with the same name results in a linking error in the HCC compiler used for ROCm/AMD.

Removing old BatchNorm & Conv files since they've been renamed.

8c5d927

Updating build path to handle ROCM

02240d9

Cleaned up the build path and created a FindHIP cmake file for settin…

6f4c403

…g up relevant hip paths.

Seperated the individual patch files to make it easier to detect issu…

414adb2

…es while building.

Removed CMakeLists hip files and fixed directory structure

c688493

Merge branch 'sending_pr' of github.com:wsttiger/pytorch into sending_pr

0044dfc

Jorghi12 requested review from apaszke, colesbury, gchanan, soumith and zdevito as code owners April 19, 2018 09:01

Jorghi12 added 3 commits April 19, 2018 02:07

Adding build pytorch amd script

b33afb6

Merged setup patch into PyTorch setup.py & cleaned a few issues

3e14982

Added information on where to download the hipify-python script.

10fb71a

Jorghi12 requested review from houseroad, jamesr66a and smessmer as code owners May 10, 2018 18:43

Jorghi12 force-pushed the sending_pr branch 2 times, most recently from ed8ef6a to c3372ee Compare May 10, 2018 19:04

Jorghi12 and others added 12 commits May 10, 2018 15:38

Rebased the specific files displayed in the PR

cb90399

Merge branch 'master' into sending_pr

e7e22ad

Fixing typo.

e2e956d

Merge branch 'sending_pr' of github.com:wsttiger/pytorch into sending_pr

6bd71e1

Change flag from "WITH_CUDA" to "NOT NO_CUDA"

f997146

Replacing "WITH_CUDA" with "NOT NO_CUDA" after the rebase.

Fix LoadHIP path

9c9edda

Updating build files after rebasing.

03a3cb1

Reorganization after cpu/gpu separation.

5fd928a

Removed HIPCC from setup.py & removed -shared extra linking args.

2d5b89a

Merge branch 'master' into sending_pr

0bcaff0

Updated CMake / Setup build to correctly link when under ROCm stack.

da5e537

Merge branch 'sending_pr' of github.com:wsttiger/pytorch into sending_pr

682138a

apaszke approved these changes May 15, 2018

View reviewed changes

setup.py Outdated

DL = Extension("torch._dl",

sources=["torch/csrc/dl.c"],

language='c',

extra_link_args=[]

This comment was marked as off-topic.

Sign in to view

Removed the unnecessary argument from Extension constructor.

e013eab

Jorghi12 added 4 commits May 14, 2018 18:10

Adding another test to be included with ROCm building.

8e941bf

Updated the setup_helpers scripts in order to get around linter error

4e2846e

Fix syntax issue

8b4e674

Solving lint issue: line too long

f5bdbf3

Jorghi12 merged commit cd86d4c into pytorch:master May 16, 2018

onnxbot added a commit to onnxbot/onnx-fb-universe that referenced this pull request May 16, 2018

[auto] Update pytorch to cd86d4c - PyTorch AMD Build Scripts (pytorch…

d470887

…/pytorch#6625) pytorch/pytorch@cd86d4c

fmassa mentioned this pull request Jun 16, 2018

OpenCL Support #488

Closed

ezyang added the open source label Jun 24, 2019

PyTorch AMD Build Scripts #6625

PyTorch AMD Build Scripts #6625

Uh oh!

Conversation

Jorghi12 commented Apr 16, 2018

Uh oh!

ezyang commented Apr 16, 2018

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

ezyang commented Apr 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apaszke left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

yf225 commented May 15, 2018

Uh oh!

Jorghi12 commented May 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ezyang commented Apr 16, 2018 •

edited

Loading

Jorghi12 commented May 16, 2018 •

edited

Loading