3X - 5X slowdown in torch.bernoulli and torch.normal on PyTorch master #11945
Description
Issue description
While debugging some perf issues in Pyro using PyTorch master, there seems to be a perf regression in the torch samplers (I checked torch.bernoulli and torch.normal). cc. @apaszke, @fritzo, @soumith
Code example
Release
In [1]: import torch
In [2]: import torch.distributions as dist
In [3]: torch.__version__
Out[3]: '0.4.1'
In [4]: z, o, b = torch.zeros(1000), torch.ones(1000), torch.ones(1000) * 0.5
In [5]: %timeit dist.Normal(z, o).sample(torch.Size([100]))
858 µs ± 10.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
In [6]: %timeit torch.normal(z, o)
8.28 µs ± 198 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [7]: %timeit dist.Bernoulli(b).sample(torch.Size([100]))
1.47 ms ± 11.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
In [8]: %timeit torch.bernoulli(b)
19.1 µs ± 230 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)Master
In [1]: import torch
In [2]: import torch.distributions as dist
In [3]: torch.__version__
Out[3]: '1.0.0a0+9cd0ae5'
In [4]: z, o, b = torch.zeros(1000), torch.ones(1000), torch.ones(1000) * 0.5
In [5]: %timeit dist.Normal(z, o).sample(torch.Size([100]))
3.7 ms ± 55.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [6]: %timeit torch.normal(z, o)
39.9 µs ± 737 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In [7]: %timeit dist.Bernoulli(b).sample(torch.Size([100]))
4.25 ms ± 77.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [8]: %timeit torch.bernoulli(b)
56 µs ± 765 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)System Info
Collecting environment information...
PyTorch version: 1.0.0a0+9cd0ae5
Is debug build: Yes
CUDA used to build PyTorch: None
OS: Mac OSX 10.13.3
GCC version: Could not collect
CMake version: version 3.12.0
Python version: 3.6
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Versions of relevant libraries:
[pip] numpy (1.15.0)
[pip] torch (1.0.0a0+9cd0ae5, /Users/npradhan/miniconda2/envs/pytorch-master/lib/python3.6/site-packages)
[pip] torchfile (0.1.0)
[pip] torchvision (0.2.1)
[conda] torch 0.5.0a0+2431eac <pip>
[conda] torch 0.5.0a0+6c3792b <pip>
[conda] torch 0.5.0a0+6660a12 <pip>
[conda] torch 0.5.0a0+35d52db <pip>
[conda] torch 1.0.0a0+9cd0ae5 <pip>
[conda] torchfile 0.1.0 <pip>
[conda] torchvision 0.2.1 <pip>