This PR is a prototype / WIP for a broad change to the way tensors are created in ATen. The fundamental motivation for this PR is to make tensor creation from C++ more like in Python, which means changing the API for factory functions like at::ones, as well as teaching ATen about construction axes beyond the backend and scalar type, meaning especially a device. This change essentially lays the ground for how tensor creation will happen in c10, and thus defines a new API, currently bolted on top of existing ATen.

This PR proposes the following:

Creation of a type encapsulating all construction axes of a tensor

At the moment, this type will contain:

• A scalar type,
• A backend,

And in the near future shall contain (and already does somewhat in this prototype):

• A device ordinal,
• A layout (strided, coo_sparse etc.)
• A requires_grad boolean.

in code:

struct TensorOptions {
        size_t device = 0;
	Layout layout = kStrided;
	ScalarType dtype = kFloat;
        Backend backend = kCPU;
	bool requires_grad = false;
};

Each of these fields have default values, such that a default-constructed TensorOptions object fully specifies a new tensor.

Furthermore, this class shall have chaining methods:

TensorOptions().device(1).dtype(kInt)

and there shall be functions with the names of fields, that return a new TensorOptions instance:

// dtype is a free function that returns a new TensorOptions
auto options = dtype(kFloat).requires_grad(true); 

Use of TensorOptions in all factory methods

The new API for tensor creation shall then be (e.g. for ones):

Tensor ones(IntList size, TensorOptions options = {}) { }

with usage examples such as

at::ones({2, 2}) // float cpu tensor
at::ones({2, 2}, kInt); // int cpu tensor
at::ones({2, 2}, type(kInt).requires_grad(true)) // int cpu tensor, which sets requires_grad for variables

New code path

This PR also implements a new code path for factory methods. For our recollection, at the moment, we create tensors via e.g.

at::ones(at::CPU(at::kFloat), {3, 3})

which dispatches to

at::CPU(at::kFloat).ones({3, 3});

which itself calls

at::native::ones(at::CPU(at::kFloat), {3, 3})

which then calls

auto tensor = at::CPU(at::kFloat).tensor({3, 3});
tensor.fill_(1);

This PR proposes to remove factory methods from Type, and skip the dispatch via Type altogether for all factory methods except tensor, which remains the fundamental source of tensors invoked by all other tensor factories:

at::ones({3, 3}, at::type(at::kFloat).device(at::CPU))

now directly calls

at::native::ones({3, 3}, at::type(at::kFloat).device(at::CPU))

which now does

// `TensorOptions::type()`  returns the type corresponding to its `backend` and `scalar_type`
auto tensor = tensor_options.type().tensor({3, 3});
tensor.fill_(1)

At the moment, I have implemented this for ones, and I think nothing broke. There should be no implications for the Python API.

@zdevito @apaszke @gchanan @colesbury @ezyang

Fixes #6285
Fixes #6286
Fixes #7735