Implement torch.util.bottleneck #5216
Conversation
|
SO COOL! :D |
torch/utils/bottleneck/__main__.py
Outdated
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
very cool! wish list for monitoring stats: gpu-util / cpu-util / analyzing cpu-bound vs ram-bound vs gpu-compute bound vs gpu-memory bound / analysis of committed gpu memory |
|
Nice tool! However presenting only the output of |
torch/utils/bottleneck/__main__.py
Outdated
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
torch/utils/bottleneck/__main__.py
Outdated
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
torch/utils/bottleneck/__main__.py
Outdated
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
Would it be possible to allow running scripts with command line arguments ? |
|
@Evpok that sounds like a good idea. I'll look into it and put it into the next iteration of this. |
This is a tool that is intended to be used as initial exploratory
debugging of bottlenecks in user scripts. Run it with
python -m torch.utils.bottleneck /path/to/source/script.py
apaszke
left a comment
apaszke
left a comment
There was a problem hiding this comment.
Looks great. It would be good to expand on CUDA profiling a bit (because it's really very complicated), but should be good to go after that.
| Due to the asynchronous nature of CUDA kernels, when running against | ||
| CUDA code, the cProfile output and CPU-mode autograd profilers may | ||
| not show correct timings. In this case, the CUDA-mode autograd | ||
| profiler is better at assigning blame to the relevant operator(s). |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
* upstream/master: (663 commits) Fix "command not found" error in perf test (pytorch#5982) add pip mkl-devel to the error message when mkl is found but mkl headers are not (pytorch#5984) Support batch LowerCholeskyTransform (pytorch#5980) Linearly interpolating upsampling fix (pytorch#5927) Store perf numbers in S3 (pytorch#5951) Modidy setup docs for Windows (pytorch#5981) Group Normalization (pytorch#5968) [distributions] Implement Power transform (pytorch#5976) Disable TestBottleneck test_cuda on Windows (pytorch#5977) Fix crash when cat-ing empty cuda tensors (pytorch#5971) Update no_unions flag for nanopb gen and update ONNX proto files (pytorch#5972) Expose gradients w.r.t. input & weight for conv1d, conv2d, conv3d in Python (pytorch#5408) Fixed non-determinate preprocessing on DataLoader (pytorch#4640) add AVX2 implementation for sigmoid function (pytorch#5010) Implement torch.util.bottleneck (pytorch#5216) Remove pragma once from cpp file (pytorch#5965) fix mvn docs (pytorch#5967) Fix incorrect rendering of Tensor.index_*_ doc examples. (pytorch#5969) Implement range for loop in script (pytorch#5827) Add windows doc (pytorch#5859) ... # Conflicts: # aten/src/TH/generic/THTensorMath.c # torch/_tensor_docs.py # torch/csrc/generic/methods/TensorCompare.cwrap
6 participants
This is a tool that is intended to be used as initial exploratory debugging of bottlenecks in user scripts. Run it with
Internally it runs the script once with the python profiler and once with the autograd profiler and prints the top 15 hits sorted by cpu time (for both profilers).
Sample Output
cc @soumith
Test Plan
Really basic tests to check that the output of
torch.util.bottleneckisn't completely empty. Taking suggestions on how to write better tests.Built the new docs page:
