Skip to content

Conversation

@sf-wind
Copy link
Contributor

@sf-wind sf-wind commented Jun 12, 2018

No description provided.

Copy link
Contributor

@soumith soumith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declarations.yaml shouldn't be included. It's auto-generated.

@soumith
Copy link
Contributor

soumith commented Jun 12, 2018

commit Skip calling ncclCommDestroy in destructor 0a6dd52 is already in pytorch master, it should be removed from this PR

@soumith
Copy link
Contributor

soumith commented Jun 12, 2018

Fix ATenOp dispatch for ops with TensorList arg is already in master via #8226 , so can be removed.

@soumith
Copy link
Contributor

soumith commented Jun 12, 2018

I believe "Resolve conflicts for tools/jit/gen_jit_dispatch.py b8e2d3a" is already in master as well. cc: @jamesr66a

This comment was marked as off-topic.

bwasti and others added 10 commits June 12, 2018 12:12
Hotfix for failues in conv_transpose
ctc_greedy_decoder same as tf's
Allow multiple callbacks per event
The motivation is to do weighted sum in HoNet/crossnet, in the next diff, I'll replace model.Add with model.WeightedSum in
honet: https://fburl.com/f4rmolg2
crossnet: https://fburl.com/v7awn8se, https://fburl.com/63filbnm
Some callers expect RunAsync to block, replicate that behavior in case of
explicit 'dag' net type
@ezyang
Copy link
Contributor

ezyang commented Jun 12, 2018

@pytorchbot retest this please

Ilia Cherniavskii and others added 8 commits June 12, 2018 12:20
Overriding dag, async_dag and async_polling with async_scheduling
Caffe thread pools currently inherit the thread names from the thread that starts them, which can be misleading. Give them an explicit name instead.
Change argument type to int64_t for shape argument of FillerOp (used in ConstantFill, XavierFill, etc)
It's not used anywhere and depends on old lua torch that conflicts with Aten. Given PT1 it's not relevant any more (though it was nice and clever code!)

#accept2ship
The multiplier needs to be non-negative, not strictly positive.
This is after 2 years and we do not seem to have a use case for this one, so
for the sake of clean API design we should potentially remove this. This would
allow us to potentially pass in arguments to optionally construct an object,
although it is indeed a little bit unclear how we can reuse existing objects if
constructor arguments are passed in. In any case, we may want to remove this
dangling feature.
Speedup generate proposals by partial_sort.

FACEBOOK:
- Saw speed improvement for training with this op.
- Yanghan benchmarked the op on a small dataset and see consistent 100% improvement on speed (6ms -> 3ms) on 420 input resolution. See next diff for details.
More parallel processing friendly for CPP version of GenerateProposals.
bwasti and others added 17 commits June 13, 2018 11:19
This reverts commit f7f434dc5c34ca6058b9765d2ef615453d2276a9

@bypass-lint

An infra SEV is better than not reverting this diff.
If you copy this password, see you in SEV Review!
@cause_a_sev_many_files
…8364)

* [IDEEP] Upgrade IDEEP version

Signed-off-by: Gu, Jinghui <jinghui.gu@intel.com>

* [IDEEP] Fix accuracy issue in conv op

Signed-off-by: Gu, Jinghui <jinghui.gu@intel.com>

* Fix build error due to lack of src in CMakeLists

Signed-off-by: Gu, Jinghui <jinghui.gu@intel.com>
* ATen fallback for ONNX export

* Move to enum

* Fix model test

* Add comment

* Address comments

BC interface
* Add hip support for caffe2 core

* Add MIOPEN header/wrapper to caffe2 core

* Add HIP device into caffe2 PB

* top level makefile change for rocm/hip

* makefile scaffolding for AMD/RocM/HIP

* Makefile scafodding for AMD/RocM/HIP; add makefile/utility for HIP files

* caffe2 PB update for AMD/ROCM HIP device

* Add AMD/RocM/Thrust dependency

* HIP threadpool update

* Fix makefile macro

* makefile fix: duplicate test/binary name

* makefile clean-up

* makefile clean-up

* add HIP operator registry

* add utilities for hip device

* Add USE_HIP to config summary

* makefile fix for BUILD_TEST

* merge latest

* Fix indentation

* code clean-up

* Guard builds without HIP and use the same cmake script as PyTorch to find HIP

* Setup rocm environment variables in build.sh (ideally should be done in the docker images)

* setup locale

* set HIP_PLATFORM

* Revert "set HIP_PLATFORM"

This reverts commit 8ec58db.

* continue the build script environment variables mess

* HCC_AMDGPU_TARGET

* Cleanup the mess, has been fixed in the lastest docker images

* Assign protobuf field hip_gpu_id a new field number for backward compatibility

* change name to avoid conflict

* Fix duplicated thread pool flag

* Refactor cmake files to not add hip includes and libs globally

* Fix the wrong usage of environment variables detection in cmake

* Add MIOPEN CNN operators

* Revert "Add MIOPEN CNN operators"

This reverts commit 6e89ad4.

* Add MIOPEN pooling operator

* Add MIOPEN activation operator

* Add MIOPEN softmax operator

* Add MIOPEN spatial batch norm operator

* Add MIOPEN loacl response normalization operator

* Add MIOPEN conv operator

* Clean-up LRN ops

* enable fp16 in MIOPEN pool ops

* Enable fp16 for MIOPEN relu op

* Enable fp16 for MIOPEN spatial batch norm op

* code clean-up

* revert float16 support

* Create Caffe2 python binding for AMD/ROCM/HIP

* Add op fallback for HIP operator

* add hip src/test files in cmake

* exclude hip src/test files

* fix python binding for hip backend

* fix MIOPEN pooling op workspace

* hack to compile miopen operators

* fix include path for MIOPEN ops

* Fix include path

* Add HIP math utilities

* Fix path for HIP math utils

* cmake fix

* Cmake fix / hipcc for hip files

* suppress hipcc warning

* cmake fix /replcae USE_HIP with USE_ROCM

* revert LoadHIP.cmake change

* fix include for thrust/cub-hip

* include path fix for conversion.h

* Updated with latest upstream changes

* clang format fixes

* Context_hip updates

* Fixed typo in rocblas handle get function

* Updated hipified math utils

* Updated math hip test util

* Updated context hip test

* Updated common_hip

* Updated net async dag for HIP

* Added MIOPEN in operator hip test

* fix

* C2 dependencies clean-up

* fix include path for building custom protobuf

* Decouple miopen pool op and conv_pool_op base

* cmake refactor

* fix operator_hip_test

* move all hip/miopen ops files into caffe2/operators/hip

* sanitize cmake

* permission issue

* remove extra parenthesis

* remove artifact from resolving merge conflict

* cont. sanitize cmake files

* fix syntax error

* sanitize conversion.h

* .

* Revert "."

This reverts commit 56020cb.

* clang-format
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
* Enable some of the ONNX backend test on broadcasting

* enable gemm broadcast
* Expose proto utils and ONNX from PyTorch libcaffe2.so

* Try to use protobuf from _C.so

* Fix ONNX proto header include

* Adjust order of imports for ONNX until nanopb goes away

* Set and use ONNX_NAMESPACE for PyTorch builds

* Show protobuf summary for all builds

* Add ONNX_NAMESPACE for cpp_build

* Statically link libprotobuf.a into libtorch.so

* Set ONNX_NAMESPACE on Windows build

* Move core/dispatch up as well

* Add /MD flag for Windows build of _C

* Potential Windows fix for ONNX and protobuf

* Add direct linkage from _C to ONNX on Windows

* Only include protobuf wrapper for PyTorch

* Pass extra_compile_args to _nvrtc ext build

* Remove installation of .a files
except ImportError:
from tools.shared.module_loader import import_module
CodeTemplate = import_module('code_template', 'aten/src/ATen/code_template.py').CodeTemplate
CodeTemplate = import_module('code_template', 'aten/src/ATen/code_template.py').CodeTemplate

This comment was marked as off-topic.

from tools.shared.module_loader import import_module
CodeTemplate = import_module('code_template', 'aten/src/ATen/code_template.py').CodeTemplate

CodeTemplate = import_module('code_template', 'aten/src/ATen/code_template.py').CodeTemplate

This comment was marked as off-topic.

except OSError:
pass
with open(os.path.join(install_dir, 'THNN.cwrap'), 'w') as f:
with open('torch/csrc/nn/THNN.cwrap', 'w') as f:

This comment was marked as off-topic.

Copy link
Contributor

@soumith soumith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge after tests pass

@sf-wind sf-wind merged commit 5b86c3a into pytorch:master Jun 13, 2018
@Yangqing
Copy link
Contributor

Are we sure the codebase is synced properly? For example, c10 folder was added 6 days ago in D8305610 (see internal), but is deleted in a more recent PR #8264 . It seems that some of the changes got lost along the way.

@Yangqing
Copy link
Contributor

cc @sf-wind @orionr FYI - Any possibility there are some commit ordering issue in the sync script?

@Yangqing
Copy link
Contributor

Did a bit more digging and it seems that tooling is fine. Basically fb internal diff landed on 3/7 afternoon (adding the c10 folder), and the github PR landed 3/7 night (strictly speaking 3/8), and because the PR added and then deleted the c10 folder during the discussion, the final landing was not aware of the c10 folder situation at all in the single squash and merge. And as a result, it was basically that a dangling c10 folder need manual deletion - sync script itself is correct.

@sf-wind sf-wind deleted the update-from-facebook branch October 24, 2018 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.