Issue description

Performing SVD using CUDA (built with MAGMA), give very different results to those given without CUDA. This is currently blocking #9052 .

Code example

>>> import torch
>>> a = torch.randn(3, 5)
>>> U, S, V = a.svd()  # Without CUDA
>>> U_c, S_c, V_c = a.cuda().svd()  # With CUDA
>>> (U - U_c.cpu()).norm()
tensor(2.0734)
>>> (S - S_c.cpu()).norm()
tensor(0.7082)
>>> (V - V_c.cpu()).norm()
tensor(2.5040)

System Info

• PyTorch or Caffe2: PyTorch
• How you installed PyTorch (conda, pip, source): source
• Build command you used (if compiling from source): python setup.py build develop
• OS: Ubuntu 16.04.2
• Python version: 3.6.5 (Miniconda)
• CUDA/cuDNN version: 8.0 / 7
• GPU models and configuration: GeForce 940M
• GCC version (if compiling from source): 5.4.0
• CMake version: 3.11.3
• Versions of any other relevant libraries:
  Versions of relevant libraries:
  [pip3] torch (0.5.0a0+ce3748d, /media/vishwak/Official/Projects/pytorch)
  [conda] magma-cuda80 2.3.0 1 pytorch
  [conda] torch 0.5.0a0+ce3748d

cc: @ssnl

EDIT by @ssnl:
eig is broken too:

>>> x = torch.randn(3,3).cuda()
>>> x.eig(True)
x.(tensor([[-0.0195,  0.7665],
        [-0.0195, -0.7665],
        [ 2.6732,  0.0000]], device='cuda:0'), tensor([[ 0.1662,  0.3178, -0.6887],
        [ 0.8708,  0.0000, -0.1837],
        [ 0.2511,  0.2238,  0.7014]], device='cuda:0'))
>>> x.cpu().eig(True)
(tensor([[ 2.6732,  0.0000],
        [-0.0195,  0.7665],
        [-0.0195, -0.7665]]), tensor([[-0.5713,  0.6626,  0.0000],
        [-0.1249, -0.3146,  0.3606],
        [ 0.8112,  0.5683,  0.0944]]))