Two differences from the NumPy implementations:

https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.take.html
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.put.html

numpy.take has an optional axis argument, which behaves like index_select. I didn't include this. We can add it later without breaking backwards compatibility.

numpy.put repeats the values if necessary to make them as long as indices. This is strange, since most other functions use broadcasting semantics. Currently, Tensor.put_ requires values to have the same length as indices. We can change this without breaking backwards compatibility.

I've added an accumulate option to put_ which handles duplicate elements (by summing them). This is necessary for the backwards of take to be correct when indices contains duplicate elements.

(The accumulate=True option is substantially slower, so accumulate is False by default).

@colesbury thanks for this work and especially the accumulate option. Are there any plans to add an axis argument to put_ as well? Specifying the index for each individual element of a tensor can take quite a lot of place in memory.

It'd be nice to have an axis argument added, since sklearn.utils.safe_indexing (used by sklearn.utils.shuffle, which is handy for sampling multiple tensors along the same indices) uses X.take(indices,axis=0) and PyTorch fails.

From what I see, numpy doesn't have an axis argument for put, and take behavior can be simulated with advanced indexing.
@alok if I understand correctly, what you are looking for can be achieved with tensor.index_select