Download as PDF, PPTX
![THEORY - LOW LEVEL API
Tensors
torch.Tensor are generalizations of a matrix that can be indexed in more
than 2 dimensions.
Creating a torch.Tensor
Tensors can be created from Python lists with the torch.Tensor() function.
import torch
vector_data = [0.0, 1.0, 0.0]
vector = torch.Tensor(vector)
matrix_data = [[1.0., 0.0, 0.0], [0.0, 1.0, 0.0]]
matrix = torch.Tensor(matrix_data)](https://image.slidesharecdn.com/pytorchfordeeplearningpractitioners-181105141725/75/PyTorch-for-Deep-Learning-Practitioners-6-2048.jpg)
![THEORY - LOW LEVEL API
Tensor Operations
Beyond Mathematical operations
You can perform indexing, slicing, joining, mutating operations on a tensors.
For example:
# Indexing
x = torch.randn(size=(1, 3))
x[0]
# Joining (Concatenation)
xx = torch.cat((x, x))
See the documentation for a complete list of the massive number of operations
available to you.](https://image.slidesharecdn.com/pytorchfordeeplearningpractitioners-181105141725/75/PyTorch-for-Deep-Learning-Practitioners-10-2048.jpg)
![THEORY - HIGH LEVEL API
Activation Functions
Softmax
Applies the Softmax function to an n-dimensional input Tensor rescaling them
so that the elements of the n-dimensional output Tensor lie in the range (0,1)
and sum to 1.
s = nn.Softmax(dim=2)
x = torch.Tensor([1, 2])
output = s(x)
See the documentation for a other layers.](https://image.slidesharecdn.com/pytorchfordeeplearningpractitioners-181105141725/75/PyTorch-for-Deep-Learning-Practitioners-17-2048.jpg)
Uploaded byBayu Aldi Yansyah
PyTorch for Deep Learning Practitioners
The document is a guide on using PyTorch for deep learning, presented by Bayu Aldi Yansyah at PyCon Indonesia in 2018. It covers both theoretical concepts and practical implementations of neural networks, detailing components like tensors, operations, layers, activation functions, and loss functions. Key takeaways emphasize PyTorch's intuitive nature and ease of use with standard Python.
More Related Content
Similar to PyTorch for Deep Learning Practitioners
![[Update] PyTorch Tutorial for NTU Machine Learing Course 2017](https://cdn.slidesharecdn.com/ss_thumbnails/pytorchtutorial-2017-1103-171103060015-thumbnail.jpg?width=640&height=640&fit=bounds)
PyTorch for Deep Learning Practitioners
- 1. Bayu Aldi Yansyah DataScientist at Kumparan github.com/pyk PyTorch for Deep Learning Practitioners PyCon Indonesia November 3-4, 2018 Kalbis Institute, Jakarta x
- 2. Who? Works 2014 - 2015Google Student Ambassador South-East Asia 2015 - 2018 Data Scientist at Sale Stock 2018 - Data Scientist at Kumparan Hobbies Author of fastText.py (900+ stars on github) And more github.com/pyk
- 3. Overview The guide isdivided into 2 main parts: 1. Theory: A brief introduction to PyTorch (concepts + code). 2. Practices: We will implement, train and deploy a basic feed-forward neural networks. We will back and forth between slide and jupyter lab.
- 4. THEORY What is PyTorch? It’sa Python-based scientific computing package that can be used for: 1. A replacement for NumPy to use the power of GPUs 2. An open source deep learning platform that provides a seamless path from research prototyping to production deployment.
- 5. THEORY Components of PyTorch TensorOps Autograd Layers Activations Loss Optim Low-level API High-level API Utilities API Data Checkpoint
- 6. THEORY - LOWLEVEL API Tensors torch.Tensor are generalizations of a matrix that can be indexed in more than 2 dimensions. Creating a torch.Tensor Tensors can be created from Python lists with the torch.Tensor() function. import torch vector_data = [0.0, 1.0, 0.0] vector = torch.Tensor(vector) matrix_data = [[1.0., 0.0, 0.0], [0.0, 1.0, 0.0]] matrix = torch.Tensor(matrix_data)
- 7. Creating a torch.Tensor Youcan create a tensor with random data and the supplied dimensionality with torch.randn(). X = torch.randn((2, 2)) y = torch.randn((2)) THEORY - LOW LEVEL API Tensors
- 8. Creating a torch.Tensor Youcan create a tensor a tensor filled with the scalar value 0 and specified dimension. Useful for bias initialization. biases = torch.zeros((3, 1, 3)) We can also specify the tensor data type using dtype property: torch.zeros((3, 1, 3), dtype=torch.int64) THEORY - LOW LEVEL API Tensors
- 9. THEORY - LOWLEVEL API Tensor Operations Mathematical operations You can perform mathematical operations on a tensors. x = torch.randn(size=(3, 2, 2)) y = torch.randn(size=(3, 2, 2)) z1 = x + y z2 = x - y z3 = x * y z4 = x / y
- 10. THEORY - LOWLEVEL API Tensor Operations Beyond Mathematical operations You can perform indexing, slicing, joining, mutating operations on a tensors. For example: # Indexing x = torch.randn(size=(1, 3)) x[0] # Joining (Concatenation) xx = torch.cat((x, x)) See the documentation for a complete list of the massive number of operations available to you.
- 11. THEORY - LOWLEVEL API Autograd Autograd helps you compute the gradient of a tensor operations automatically. It is very useful for back propagation algorithm. For example, suppose: How to compute gradient of s w.r.t element of w?
- 12. THEORY - LOWLEVEL API Autograd Easy. You don’t need to do it manually. x = torch.randn(size=(3,), requires_grad=True) w = torch.randn(size=(3,), requires_grad=True) z = x * w s = z.sum()
- 13. THEORY - LOWLEVEL API Autograd s.backward() assert(w.grad.equal(x)) DONE.
- 14. THEORY - HIGHLEVEL API Layers PyTorch layers is a python class that represents a neural network layers. It built on top of Tensor, Ops and Autograd. Available layers: 1. Convolution Layers 2. Pooling Layers 3. Padding Layers 4. Normalization Layers 5. Recurrent layers 6. Linear layers 7. Dropout Layers 8. Embedding Layers
- 15. THEORY - HIGHLEVEL API Layers Linear Layers Applies a linear transformation to the incoming tensor. from torch import nn lin = nn.Linear(40, 50) x = torch.randn(size=(10, 40)) output = lin(x) See the documentation for a other layers.
- 16. THEORY - HIGHLEVEL API Activation Functions In PyTorch there are already exists a bunch of pre-defined activation functions that we can use. For example: 1. ReLU (torch.nn.ReLU) 2. Sigmoid (torch.nn.Sigmoid) 3. Tanh (torch.nn.Tanh) 4. Softmax (torch.nn.Softmax)
- 17. THEORY - HIGHLEVEL API Activation Functions Softmax Applies the Softmax function to an n-dimensional input Tensor rescaling them so that the elements of the n-dimensional output Tensor lie in the range (0,1) and sum to 1. s = nn.Softmax(dim=2) x = torch.Tensor([1, 2]) output = s(x) See the documentation for a other layers.
- 18. THEORY - HIGHLEVEL API Loss Functions In PyTorch there are exists pre-defined loss function that we can use. For example: 1. MSELoss (torch.nn.MSELoss) 2. CrossEntropyLoss (torch.nn.CrossEntropyLoss)
- 19. THEORY - HIGHLEVEL API Loss Functions Cross Entropy Loss It is useful when training a classification problem with N classes. loss = nn.CrossEntropyLoss() y_hat = torch.randn(3, 5, requires_grad=True) y = torch.empty(3, dtype=torch.long).random_(5) output = loss(y_hat, y)
- 20. PRACTICES Implement, Train &Deploy a Model Enough theories, let’s practice!
- 21. Key Takeaways 1. PyTorchis very intuitive to use. 2. There is no cognitive overhead, it just a plain python.
- 22. Bayu Aldi Yansyah DataScientist at Kumparan github.com/pyk PyCon Indonesia November 3-4, 2018 Kalbis Institute, Jakarta Questions? Email me: bay@machinelearning.id