Mohamed Loey, profile picture
Uploaded byMohamed Loey
PDF, PPTX15,585 views

Convolutional Neural Network Models - Deep Learning

The document provides an overview of Convolutional Neural Network (CNN) models developed for image classification, highlighting notable models such as AlexNet, ZFNet, VGGNet, GoogLeNet, and ResNet. It outlines the architecture, training processes, and performance metrics of these models, including their respective top-5 error rates in the ImageNet Large Scale Visual Recognition Challenge. Additionally, it covers specific training techniques and innovations employed by each model to improve accuracy and efficiency.

Embed presentation

Download as PDF, PPTX
CNN Models Convolutional Neural Network Models
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  Convolutional Neural Network (CNN)is a multi-layer neural network  Convolutional Neural Network is comprised of one or more convolutional layers (often with a pooling layers) and then followed by one or more fully connected layers.
CNN Models  Convolutional layer acts as a feature extractor that extracts features of the inputs such as edges, corners , endpoints.
CNN Models  Pooling layer reduces the resolution of the image that reduce the precision of the translation (shift and distortion) effect.
CNN Models  fully connected layer have full connections to all activations in the previous layer.  Fully connect layer act as classifier.
CNN Models Output Image = ( ( (ImageSize+2*Padding)- KernalSize )/ Stride) +1
CNN Models  Conv 3x3 with stride=1,padding=0 6x6 Image 4x4
CNN Models  Conv 3x3 with stride=1,padding=1 4x4 Image 4x4
CNN Models  Conv 3x3 with stride=2,padding=0 7x7 Image 3x3
CNN Models  Conv 3x3 with stride=2,padding=1 5x5 Image 3x3
CNN Models  MaxPooling 2x2 with stride=2 4x4 Image 2x2
CNN Models  MaxPooling 3x3 with stride=2 7x7 Image 3x3
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  ImageNet Large Scale Visual Recognition Challenge is image classification challenge to create model that can correctly classify an input image into 1,000 separate object categories. Models are trained on 1.2 million training images with another 50,000 images for validation and 150,000 images for testing
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  AlexNet achieve on ILSVRC 2012 competition 15.3% Top-5 error rate compare to 26.2% achieved by the second best entry.  AlexNet using batch stochastic gradient descent on training, with specific values for momentum and weight decay.  AlexNet implement dropout layers in order to combat the problem of overfitting to the training data.
CNN Models Image Conv1 Pool1 Conv2 Pool2 Conv3 Conv4 Conv5 Pool3 FC1 FC2 FC3  AlexNet has 8 layers without count pooling layers.  AlexNet use ReLU for the nonlinearity functions  AlexNet trained on two GTX 580 GPUs for five to six days
CNN Models Image 227x227x3 Conv11-96 Maxpool Conv5-256 MaxpoolConv3-384Conv3-384Conv3-256 Maxpool FC-4096 FC-4096 FC-1000
CNN Models  AlexNet Model
CNN Models  Layer 0: Input image  Size: 227 x 227 x 3  Memory: 227 x 227 x 3
CNN Models  Layer 0: 227 x 227 x 3  Layer 1: Convolution with 96 filters, size 11×11, stride 4, padding 0  Outcome Size= 55 x 55 x 96  (227-11)/4 + 1 = 55 is size of outcome  Memory: 55 x 55 x 96 x 3 (because of ReLU & LRN(Local Response Normalization))  Weights (parameters) : 11 x 11 x 3 x 96
CNN Models  Layer 1: 55 x 55 x 96  Layer 2: Max-Pooling with 3×3 filter, stride 2  Outcome Size= 27 x 27 x 96  (55 – 3)/2 + 1 = 27 is size of outcome  Memory: 27 x 27 x 96
CNN Models  Layer 2: 27 x 27 x 96  Layer 3: Convolution with 256 filters, size 5×5, stride 1, padding 2  Outcome Size = 27 x 27 x 256  original size is restored because of padding  Memory: 27 x 27 x 256 x 3 (because of ReLU and LRN)  Weights: 5 x 5 x 96 x 256
CNN Models  Layer 3: 27 x 27 x 256  Layer 4: Max-Pooling with 3×3 filter, stride 2  Outcome Size = 13 x 13 x 256  (27 – 3)/2 + 1 = 13 is size of outcome  Memory: 13 x 13 x 256
CNN Models  Layer 4: 13 x 13 x 256  Layer 5: Convolution with 384 filters, size 3×3, stride 1, padding 1  Outcome Size = 13 x 13 x 384  the original size is restored because of padding (13+2 -3)/1 +1 =13  Memory: 13 x 13 x 384 x 2 (because of ReLU)  Weights: 3 x 3 x 256 x 384
CNN Models  Layer 5: 13 x 13 x 384  Layer 6: Convolution with 384 filters, size 3×3, stride 1, padding 1  Outcome Size = 13 x 13 x 384  the original size is restored because of padding  Memory: 13 x 13 x 384 x 2 (because of ReLU)  Weights: 3 x 3 x 384 x 384
CNN Models  Layer 6: 13 x 13 x 384  Layer 7: Convolution with 256 filters, size 3×3, stride 1, padding 1  Outcome Size = 13 x 13 x 256  the original size is restored because of padding  Memory: 13 x 13 x 256 x 2 (because of ReLU)  Weights: 3 x 3 x 384 x 256
CNN Models  Layer 7: 13 x 13 x 256  Layer 8: Max-Pooling with 3×3 filter, stride 2  Outcome Size = 6 x 6 x 256  (13 – 3)/2 + 1 = 6 is size of outcome  Memory: 6 x 6 x 256
CNN Models  Layer 8: 6x6x256=9216 pixels are fed to FC  Layer 9: Fully Connected with 4096 neuron  Memory: 4096 x 3 (because of ReLU and Dropout)  Weights: 4096 x (6 x 6 x 256)
CNN Models  Layer 9: Fully Connected with 4096 neuron  Layer 10: Fully Connected with 4096 neuron  Memory: 4096 x 3 (because of ReLU and Dropout)  Weights: 4096 x 4096
CNN Models  Layer 10: Fully Connected with 4096 neuron  Layer 11: Fully Connected with 1000 neurons  Memory: 1000  Weights: 4096 x 1000
CNN Models  Total (label and softmax not included)  Memory: 2.24 million  Weights: 62.37 million
CNN Models  first use of ReLU  Alexnet used Norm layers  Alexnet heavy used data augmentation  Alexnet use dropout 0.5  Alexnet batch size is 128  Alexnet used SGD Momentum 0.9  Alexnet used learning rate 1e-2, reduced by 10
CNN Models [227x227x3] INPUT [55x55x96] CONV1 : 96 11x11 filters at stride 4, pad 0 27x27x96] MAX POOL1 : 3x3 filters at stride 2 [27x27x96] NORM1: Normalization layer [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 [13x13x256] MAX POOL2: 3x3 filters at stride 2 [13x13x256] NORM2: Normalization layer
CNN Models [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 [6x6x256] MAX POOL3: 3x3 filters at stride 2 [4096] FC6: 4096 neurons [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons
CNN Models  Implement AlexNet using TFLearn
CNN Models
CNN Models
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  ZFNet the winner of the competition ILSVRC 2013 with 14.8% Top-5 error rate  ZFNet built by Matthew Zeiler and Rob Fergus  ZFNet has the same global architecture as Alexnet, that is to say 5 convolutional layers, two fully connected layers and an output softmax one. The differences are for example better sized convolutional kernels.
CNN Models  ZFNet used filters of size 7x7 and a decreased stride value, instead of using 11x11 sized filters in the first layer (which is what AlexNet implemented).  ZFNet trained on a GTX 580 GPU for twelve days.  Developed a visualization technique named Deconvolutional Network “deconvnet” because it maps features to pixels.
CNN Models AlexNet but: • CONV1: change from (11x11 stride 4) to (7x7 stride 2) • CONV3,4,5: instead of 384, 384, 256 filters use 512, 1024, 512
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  Keep it deep. Keep it simple.  VGGNet the runner up of the competition ILSVRC 2014 with 7.3% Top-5 error rate.  VGGNet use of only 3x3 sized filters is quite different from AlexNet’s 11x11 filters in the first layer and ZFNet’s 7x7 filters.  two 3x3 conv layers have an effective receptive field of 5x5  Three 3x3 conv layers have an effective receptive field of 7x7  VGGNet trained on 4 Nvidia Titan Black GPUs for two to three weeks
CNN Models  Interesting to notice that the number of filters doubles after each maxpool layer. This reinforces the idea of shrinking spatial dimensions, but growing depth.  VGGNet used scale jittering as one data augmentation technique during training  VGGNet used ReLU layers after each conv layer and trained with batch gradient descent
CNN Models Image Conv Conv Pool Conv Conv Pool Conv Conv Conv Pool Conv Conv Conv Pool Conv Conv Conv Pool FC FC FC Image Low Level Feature Mid Level Feature High Level Feature Classifier
CNN Models Input 224x224x3 Conv3-64 Conv3-64 Maxpool Conv3-128 Conv3-128 MaxpoolConv3-256Conv3-256Conv3-256MaxpoolConv3-512 Conv3-512 Conv3-512 Maxpool Conv3-512 Conv3-512 Conv3-512 MaxpoolFC-4096FC-4096FC-1000VGGNet 16
CNN Models VGGNet 16
CNN Models Input 224x224x3 Conv3-64 Conv3-64 Maxpool Conv3-128 Conv3-128 Maxpool Conv3-256Conv3-256Conv3-256Conv3-256MaxpoolConv3-512Conv3-512 Conv3-512 Conv3-512 Maxpool Conv3-512 Conv3-512 Conv3-512 Conv3-512 MaxpoolFC-4096FC-4096FC-1000 VGGNet 19
CNN Models  Implement VGGNet16 using TFLearn
CNN Models
CNN Models
CNN Models
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  GoogleNet is the winner of the competition ILSVRC 2014 with 6.7% Top-5 error rate.  GoogleNet Trained on “a few high-end GPUs with in a week”  GoogleNet uses 12x fewer parameters than AlexNet  GoogleNet use an average pool instead of fully connected layers, to go from a 7x7x1024 volume to a 1x1x1024 volume. This saves a huge number of parameters.
CNN Models  GoogleNet used 9 Inception modules in the whole architecture  This 1x1 convolutions (bottleneck convolutions) allow to control/reduce the depth dimension which greatly reduces the number of used parameters due to removal of redundancy of correlated filters.  GoogleNet has 22 Layers deep network
CNN Models  GoogleNet use an average pool instead of using FC-Layer, to go from a 7x7x1024 volume to a 1x1x1024 volume. This saves a huge number of parameters.  GoogleNet use inexpensive Conv1 to compute reduction before the expensive Conv3 and Conv5  Conv1 follow by Relu to reduce overfitting
CNN Models  Inception module
CNN Models Input 224x224x3 Conv7/2-64 Maxpool3/2 Conv1 Conv3/1- 192 Maxpool3/2 Inception3a 256 Inception3b 480 Maxpool3/2 Inception4a 512 Inception4b 512 Inception4c 512 Inception4d 528 Inception4e 832 Maxpool3/2 Inception5a 832 Inception5b 1024 Avgpool7/1 Dropout 40% FC-1000 Softmax- 1000GoogleNet
CNN Models
CNN Models Type Size/ Stride Output Depth Conv1 # Conv3 Conv3 # Conv5 Conv5 Pool Param Ops Conv 7x7/2 112x112x64 1 - - - - - - 2.7K 34M Maxpool 3x3/2 56x56x64 0 - - - - - - - - Conv 3x3/1 56x56x192 2 - 64 192 - - - 112K 360M Maxpool 3x3/2 28x28x192 0 - - - - - - - - Inception 3a - 28x28x256 2 64 96 128 16 32 32 159K 128M Inception 3b - 28x28x480 2 128 128 192 32 96 64 380K 304M Maxpool 3x3/2 14x14x480 0 - - - - - - - - Inception 4a - 14x14x512 2 192 96 208 16 48 64 364K 73M Inception 4b - 14x14x512 2 160 112 224 24 64 64 437K 88M Inception 4c - 14x14x512 2 128 128 256 24 64 64 463K 100M Inception 4d - 14x14x528 2 112 144 288 32 64 64 580K 119M
CNN Models Type Size/ Stride Output Depth Conv1 # Conv3 Conv3 # Conv5 Conv5 Pool Param Ops Inception 4e - 14x14x832 2 256 160 320 32 128 128 840K 170M Maxpool 3x3/2 7x7x832 0 - - - - - - - - Inception 5a - 7x7x832 2 256 160 320 32 128 128 1072K 54M Inception 5b - 7x7x1024 2 384 192 384 48 128 128 1388K 71M Avgpool 7x7/1 1x1x1024 0 - - - - - - - - Dropout .4 - 1x1x1024 0 - - - - - - - - Linear - 1x1x1024 1 - - - - - - 1000K 1M Softmax - 1x1x1024 0 - - - - - - - - Total Layers 22
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models  ResNet the winner of the competition ILSVRC 2015 with 3.6% Top-5 error rate.  ResNet mainly inspired by the philosophy of VGGNet.  ResNet proposed a residual learning approach to ease the difficulty of training deeper networks. Based on the design ideas of Batch Normalization (BN), small convolutional kernels.  ResNet is a new 152 layer network architecture.  ResNet Trained on an 8 GPU machine for two to three weeks
CNN Models  Residual network  Keys:  No max pooling  No hidden fc  No dropout  Basic design (VGG-style)  All 3x3 conv (almost)  Batch normalization
CNN Models Conv Layers Preserving base information can treat perturbation
CNN Models  Residual block
CNN Models  Residual Bottleneck consist of a 1×1 layer for reducing dimension, a 3×3 layer, and a 1×1 layer for restoring dimension.
CNN Models Image Conv7/2- 64 Pool/2 Conv3-64 Conv3-64 Conv3-64 Conv3-64 Conv3-64 Conv3-64 Conv3/2- 128 Conv3-128Conv3-128Conv3-128Conv3-128Conv3-128Conv3-128 Conv3-128 Conv3/2- 256 Conv3-256 Conv3-256 Conv3-256 Conv3-256 Conv3-256 Conv3-256 Conv3-256Conv3-256Conv3-256Conv3-256Conv3-256 Conv3/2- 512 Conv3-512Conv3-512 Conv3-512 Conv3-512 Conv3-512 Avg pool FC-1000 ResNet 34
CNN Models Image Conv7/2-64 Pool/2 2Conv3-64 2Conv3-64 2Conv3-64 2Conv3/2- 128 2Conv3-1282Conv3-1282Conv3-128 2Conv3/2- 256 2Conv3-256 2Conv3-256 2Conv3-256 2Conv3-256 2Conv3-256 2Conv3/2- 512 2Conv3-512 2Conv3-512Avg poolFC-1000ResNet 34
CNN Models  ResNet Model
CNN Models Layer Output 18-Layer 34-Layer 50-Layer 101-Layer 152-Layer Conv-1 112x112 7x7/2-64 Conv-2 56x56 3x3 Maxpooling/2 𝟐𝐱 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱 𝟏𝐱𝟏, 𝟔𝟒 𝟑𝐱𝟑𝐱𝟔𝟒 𝟏𝐱𝟏𝐱𝟐𝟓𝟔 𝟑𝐱 𝟏𝐱𝟏, 𝟔𝟒 𝟑𝐱𝟑𝐱𝟔𝟒 𝟏𝐱𝟏𝐱𝟐𝟓𝟔 𝟑𝐱 𝟏𝐱𝟏, 𝟔𝟒 𝟑𝐱𝟑𝐱𝟔𝟒 𝟏𝐱𝟏𝐱𝟐𝟓𝟔 Conv-3 28x28 𝟐𝐱 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟒𝐱 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟒𝐱 𝟏𝐱𝟏, 𝟏𝟐𝟖 𝟑𝐱𝟑𝐱𝟏𝟐𝟖 𝟏𝐱𝟏𝐱𝟓𝟏𝟐 𝟒𝐱 𝟏𝐱𝟏, 𝟏𝟐𝟖 𝟑𝐱𝟑𝐱𝟏𝟐𝟖 𝟏𝐱𝟏𝐱𝟓𝟏𝟐 𝟖𝐱 𝟏𝐱𝟏, 𝟏𝟐𝟖 𝟑𝐱𝟑𝐱𝟏𝟐𝟖 𝟏𝐱𝟏𝐱𝟓𝟏𝟐 Conv-4 14x14 𝟐𝐱 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟔𝐱 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟔𝐱 𝟏𝐱𝟏, 𝟐𝟓𝟔 𝟑𝐱𝟑𝐱𝟐𝟓𝟔 𝟏𝐱𝟏𝐱𝟏𝟎𝟐𝟒 𝟐𝟑𝐱 𝟏𝐱𝟏, 𝟐𝟓𝟔 𝟑𝐱𝟑𝐱𝟐𝟓𝟔 𝟏𝐱𝟏𝐱𝟏𝟎𝟐𝟒 𝟑𝟔𝐱 𝟏𝐱𝟏, 𝟐𝟓𝟔 𝟑𝐱𝟑𝐱𝟐𝟓𝟔 𝟏𝐱𝟏𝐱𝟏𝟎𝟐𝟒 Conv-5 7x7 𝟐𝐱 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱 𝟏𝐱𝟏, 𝟓𝟏𝟐 𝟑𝐱𝟑𝐱𝟓𝟏𝟐 𝟏𝐱𝟏𝐱𝟐𝟎𝟒𝟖 𝟑𝐱 𝟏𝐱𝟏, 𝟓𝟏𝟐 𝟑𝐱𝟑𝐱𝟓𝟏𝟐 𝟏𝐱𝟏𝐱𝟐𝟎𝟒𝟖 𝟑𝐱 𝟏𝐱𝟏, 𝟓𝟏𝟐 𝟑𝐱𝟑𝐱𝟓𝟏𝟐 𝟏𝐱𝟏𝐱𝟐𝟎𝟒𝟖 1x1 Avgpool-FC1000-Softmax Flops 𝟏. 𝟖𝐱𝟏𝟎 𝟗 𝟑. 𝟔𝐱𝟏𝟎 𝟗 𝟑. 𝟖𝐱𝟏𝟎 𝟗 𝟕. 𝟔𝐱𝟏𝟎 𝟗 𝟏𝟏. 𝟑𝐱𝟏𝟎 𝟗
CNN Models  Implement ResNet using TFLearn
CNN Models
CNN Models
CNN Models
CNN Models Convolutional Neural Network ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
CNN Models 26.2 15.3 14.8 7.3 6.7 3.6 0 5 10 15 20 25 30 Before 2012 AlexNet 2012 ZFNet 2013 VGGNet 2014 GoogleNet 2014 ResNet 2015
CNN Models
CNN Models facebook.com/mloey mohamedloey@gmail.com twitter.com/mloey linkedin.com/in/mloey mloey@fci.bu.edu.eg mloey.github.io
CNN Models www.YourCompany.com © 2020 Companyname PowerPoint Business Theme. All Rights Reserved. THANKS FOR YOUR TIME

More Related Content

PPTX
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
bySimplilearn
 
PDF
Deep Learning - Convolutional Neural Networks
byChristian Perone
 
PPTX
Machine Learning - Convolutional Neural Network
byRichard Kuo
 
PPTX
Convolutional Neural Network and Its Applications
byKasun Chinthaka Piyarathna
 
PPTX
Introduction to CNN
byShuai Zhang
 
PPTX
Convolution Neural Network (CNN)
byBasit Rafiq
 
PPTX
Image classification using cnn
bySumeraHangi
 
PPTX
CONVOLUTIONAL NEURAL NETWORK
byMd Rajib Bhuiyan
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
bySimplilearn
 
Deep Learning - Convolutional Neural Networks
byChristian Perone
 
Machine Learning - Convolutional Neural Network
byRichard Kuo
 
Convolutional Neural Network and Its Applications
byKasun Chinthaka Piyarathna
 
Introduction to CNN
byShuai Zhang
 
Convolution Neural Network (CNN)
byBasit Rafiq
 
Image classification using cnn
bySumeraHangi
 
CONVOLUTIONAL NEURAL NETWORK
byMd Rajib Bhuiyan
 

What's hot

PDF
Convolutional neural network
byYan Xu
 
PPTX
Convolutional Neural Network (CNN) - image recognition
byYUNG-KUEI CHEN
 
PPTX
Convolutional neural network
byMojammilHusain
 
PPTX
CNN and its applications by ketaki
byKetaki Patwari
 
PDF
Convolutional Neural Networks (CNN)
byGaurav Mittal
 
PPTX
CNN Tutorial
bySungjoon Choi
 
PPTX
Convolutional Neural Network
byVignesh Suresh
 
PPTX
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
PPTX
Convolutional Neural Networks
byAshray Bhandare
 
PPTX
Deep Learning - CNN and RNN
byAshray Bhandare
 
PPTX
Convolutional Neural Network (CNN)
byMuhammad Haroon
 
PPTX
Convolutional Neural Networks
bymilad abbasi
 
PPTX
Image classification using CNN
byNoura Hussein
 
PDF
Image segmentation with deep learning
byAntonio Rueda-Toicen
 
PDF
Resnet
byashwinjoseph95
 
PPTX
Deep learning
byRajgupta258
 
PPTX
Deep neural networks
bySi Haem
 
PDF
PR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
byJinwon Lee
 
PPTX
Introduction to Deep learning
byleopauly
 
PPTX
Activation functions and Training Algorithms for Deep Neural network
byGayatri Khanvilkar
 
Convolutional neural network
byYan Xu
 
Convolutional Neural Network (CNN) - image recognition
byYUNG-KUEI CHEN
 
Convolutional neural network
byMojammilHusain
 
CNN and its applications by ketaki
byKetaki Patwari
 
Convolutional Neural Networks (CNN)
byGaurav Mittal
 
CNN Tutorial
bySungjoon Choi
 
Convolutional Neural Network
byVignesh Suresh
 
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
Convolutional Neural Networks
byAshray Bhandare
 
Deep Learning - CNN and RNN
byAshray Bhandare
 
Convolutional Neural Network (CNN)
byMuhammad Haroon
 
Convolutional Neural Networks
bymilad abbasi
 
Image classification using CNN
byNoura Hussein
 
Image segmentation with deep learning
byAntonio Rueda-Toicen
 
Resnet
byashwinjoseph95
 
Deep learning
byRajgupta258
 
Deep neural networks
bySi Haem
 
PR-108: MobileNetV2: Inverted Residuals and Linear Bottlenecks
byJinwon Lee
 
Introduction to Deep learning
byleopauly
 
Activation functions and Training Algorithms for Deep Neural network
byGayatri Khanvilkar
 

Viewers also liked

PDF
Deep Learning - Overview of my work II
byMohamed Loey
 
PDF
Algorithms Lecture 4: Sorting Algorithms I
byMohamed Loey
 
PDF
Algorithms Lecture 2: Analysis of Algorithms I
byMohamed Loey
 
PDF
Algorithms Lecture 5: Sorting Algorithms II
byMohamed Loey
 
PDF
Algorithms Lecture 3: Analysis of Algorithms II
byMohamed Loey
 
PDF
Computer Security Lecture 5: Simplified Advanced Encryption Standard
byMohamed Loey
 
PPSX
C++ Programming Language
byMohamed Loey
 
PDF
Computer Security Lecture 7: RSA
byMohamed Loey
 
PDF
PMP Lecture 1: Introduction to Project Management
byMohamed Loey
 
Deep Learning - Overview of my work II
byMohamed Loey
 
Algorithms Lecture 4: Sorting Algorithms I
byMohamed Loey
 
Algorithms Lecture 2: Analysis of Algorithms I
byMohamed Loey
 
Algorithms Lecture 5: Sorting Algorithms II
byMohamed Loey
 
Algorithms Lecture 3: Analysis of Algorithms II
byMohamed Loey
 
Computer Security Lecture 5: Simplified Advanced Encryption Standard
byMohamed Loey
 
C++ Programming Language
byMohamed Loey
 
Computer Security Lecture 7: RSA
byMohamed Loey
 
PMP Lecture 1: Introduction to Project Management
byMohamed Loey
 

Similar to Convolutional Neural Network Models - Deep Learning

PDF
Lecture 5: Convolutional Neural Network Models
byMohamed Loey
 
PPTX
Image Classification using deep learning
byAsma-AH
 
PDF
Hardware Acceleration for Machine Learning
byCastLabKAIST
 
PDF
Case Study of Convolutional Neural Network
byNamHyuk Ahn
 
PPTX
intro-to-cnn-April_2020.pptx
byssuser3aa461
 
PDF
Cs231n 2017 lecture9 CNN Architecture
byYanbin Kong
 
PDF
Modern Convolutional Neural Network techniques for image segmentation
byGioele Ciaparrone
 
PPTX
ImageNet classification with deep convolutional neural networks(2012)
byWoochulShin10
 
PPTX
Computer Vision for Beginners
bySanghamitra Deb
 
PDF
convolutional_neural_networks in deep learning
byssusere5ddd6
 
PPTX
CNN-AlexNet-ResNet presentation describing their architectures
bylunalovegood66
 
PPTX
CNN LeNet5 Architecture: Neural Networks
byDrRPonnusamyCIT
 
PDF
lecture16.pdf000000000000000000000000000
bythed3942
 
PPTX
Tìm hiểu về CNN và ResNet | Computer Vision
byNgAnhTun27
 
PPTX
A Survey of Convolutional Neural Networks
byRimzim Thube
 
PPTX
Introduction to computer vision with Convoluted Neural Networks
byMarcinJedyk
 
PPTX
Introduction to computer vision
byMarcin Jedyk
 
PDF
cs231n_2018_lecture09_ThiGiacMay_SPKT.pdf
bycaoquang10062004
 
PDF
Introduction to CNN with Application to Object Recognition
byArtifacia
 
PDF
Finding the best solution for Image Processing
byTech Triveni
 
Lecture 5: Convolutional Neural Network Models
byMohamed Loey
 
Image Classification using deep learning
byAsma-AH
 
Hardware Acceleration for Machine Learning
byCastLabKAIST
 
Case Study of Convolutional Neural Network
byNamHyuk Ahn
 
intro-to-cnn-April_2020.pptx
byssuser3aa461
 
Cs231n 2017 lecture9 CNN Architecture
byYanbin Kong
 
Modern Convolutional Neural Network techniques for image segmentation
byGioele Ciaparrone
 
ImageNet classification with deep convolutional neural networks(2012)
byWoochulShin10
 
Computer Vision for Beginners
bySanghamitra Deb
 
convolutional_neural_networks in deep learning
byssusere5ddd6
 
CNN-AlexNet-ResNet presentation describing their architectures
bylunalovegood66
 
CNN LeNet5 Architecture: Neural Networks
byDrRPonnusamyCIT
 
lecture16.pdf000000000000000000000000000
bythed3942
 
Tìm hiểu về CNN và ResNet | Computer Vision
byNgAnhTun27
 
A Survey of Convolutional Neural Networks
byRimzim Thube
 
Introduction to computer vision with Convoluted Neural Networks
byMarcinJedyk
 
Introduction to computer vision
byMarcin Jedyk
 
cs231n_2018_lecture09_ThiGiacMay_SPKT.pdf
bycaoquang10062004
 
Introduction to CNN with Application to Object Recognition
byArtifacia
 
Finding the best solution for Image Processing
byTech Triveni
 

More from Mohamed Loey

PPTX
Lecture 3: Convolutional Neural Networks
byMohamed Loey
 
PDF
Algorithms Lecture 1: Introduction to Algorithms
byMohamed Loey
 
PDF
Lecture 1: Deep Learning for Computer Vision
byMohamed Loey
 
PDF
Algorithms Lecture 7: Graph Algorithms
byMohamed Loey
 
PDF
Algorithms Lecture 6: Searching Algorithms
byMohamed Loey
 
PDF
Lecture 4: Deep Learning Frameworks
byMohamed Loey
 
PDF
Algorithms Lecture 8: Pattern Algorithms
byMohamed Loey
 
PDF
Lecture 2: Artificial Neural Network
byMohamed Loey
 
PDF
Computer Security - CCNA Security - Lecture 1
byMohamed Loey
 
PDF
Computer Security Lecture 4: Block Ciphers and the Data Encryption Standard
byMohamed Loey
 
PDF
Computer Security Lecture 1: Overview
byMohamed Loey
 
PDF
Computer Security Lecture 2: Classical Encryption Techniques 1
byMohamed Loey
 
PDF
Computer Security - CCNA Security - Lecture 2
byMohamed Loey
 
PDF
Computer Security Lecture 3: Classical Encryption Techniques 2
byMohamed Loey
 
PDF
Lecture 4: How it Works: Convolutional Neural Networks
byMohamed Loey
 
PDF
Computer Security Lecture 4.1: DES Supplementary Material
byMohamed Loey
 
PDF
Lecture 6: Deep Learning Applications
byMohamed Loey
 
PDF
PMP Lecture 4: Project Integration Management
byMohamed Loey
 
PDF
PMP Lecture 3: Project Management Processes
byMohamed Loey
 
PDF
Design of an Intelligent System for Improving Classification of Cancer Diseases
byMohamed Loey
 
Lecture 3: Convolutional Neural Networks
byMohamed Loey
 
Algorithms Lecture 1: Introduction to Algorithms
byMohamed Loey
 
Lecture 1: Deep Learning for Computer Vision
byMohamed Loey
 
Algorithms Lecture 7: Graph Algorithms
byMohamed Loey
 
Algorithms Lecture 6: Searching Algorithms
byMohamed Loey
 
Lecture 4: Deep Learning Frameworks
byMohamed Loey
 
Algorithms Lecture 8: Pattern Algorithms
byMohamed Loey
 
Lecture 2: Artificial Neural Network
byMohamed Loey
 
Computer Security - CCNA Security - Lecture 1
byMohamed Loey
 
Computer Security Lecture 4: Block Ciphers and the Data Encryption Standard
byMohamed Loey
 
Computer Security Lecture 1: Overview
byMohamed Loey
 
Computer Security Lecture 2: Classical Encryption Techniques 1
byMohamed Loey
 
Computer Security - CCNA Security - Lecture 2
byMohamed Loey
 
Computer Security Lecture 3: Classical Encryption Techniques 2
byMohamed Loey
 
Lecture 4: How it Works: Convolutional Neural Networks
byMohamed Loey
 
Computer Security Lecture 4.1: DES Supplementary Material
byMohamed Loey
 
Lecture 6: Deep Learning Applications
byMohamed Loey
 
PMP Lecture 4: Project Integration Management
byMohamed Loey
 
PMP Lecture 3: Project Management Processes
byMohamed Loey
 
Design of an Intelligent System for Improving Classification of Cancer Diseases
byMohamed Loey
 

Recently uploaded

DOCX
TOXICITY AND ITS MANAGEMENT 6th sem unit 5, PHARMACOLOGY-III B. PHARMACY
byjagdeepsinghynr7
 
PDF
Prescription Writing- Elements, Parts, and Exercises
byDr. Ishita Agarwal
 
PDF
State: Meaning, Origin, Sovereignty & Forms of Government – A Comprehensive O...
byDr Raja Mohammed T
 
PDF
Handwritten notes of Toxicity 1. Genotoxicity 2 Carcinogenicity 3 Teratogenic...
byjagdeepsinghynr7
 
PPTX
ELEMENTS OF COMMUNICATION (UNIT 2) .pptx
byPOOJA KARVANDE
 
PDF
Digital Wellness in University Communities: Libraries as Guardians of Healthy...
bySylvester Ebhonu
 
PPTX
Cost of Capital - Cost of Equity, Cost of debenture, Cost of Preference share...
bySundar B N
 
PDF
The Pity of War: Form, Fragment, and the Artificial Echo | Understanding War ...
byNidhi Pandya
 
PDF
Orchard Floor Managment.pdf Orchard Floor Management
bybisensharad
 
PPTX
The Art Pastor's Guide to the Liturgical Calendar
bySteve Thomason
 
PPTX
Rectal Surgery in Senior Citiizens .pptx
byJohn Thanakumar
 
PDF
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
PDF
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
PDF
1ST APPLICATION FOR ANNULMENT (4)8787666.pdf
bykonstantinantountoum1
 
PDF
Projecte de la porta de la classe de primer A: Mar i cel.
byeduardrubio1
 
PDF
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
PPTX
TAMIS & TEMS - HOW, WHY and THE STEPS IN PROCTOLOGY
byJohn Thanakumar
 
PPTX
Limpitlaw "Licensing: From Mindset to Milestones"
byNational Information Standards Organization (NISO)
 
PPTX
Details of Epithelial and Connective Tissue.pptx
byAshish Umale
 
PPTX
ICH Harmonization A Global Pathway to Unified Drug Regulation.pptx
byDr. Smita Kumbhar
 
TOXICITY AND ITS MANAGEMENT 6th sem unit 5, PHARMACOLOGY-III B. PHARMACY
byjagdeepsinghynr7
 
Prescription Writing- Elements, Parts, and Exercises
byDr. Ishita Agarwal
 
State: Meaning, Origin, Sovereignty & Forms of Government – A Comprehensive O...
byDr Raja Mohammed T
 
Handwritten notes of Toxicity 1. Genotoxicity 2 Carcinogenicity 3 Teratogenic...
byjagdeepsinghynr7
 
ELEMENTS OF COMMUNICATION (UNIT 2) .pptx
byPOOJA KARVANDE
 
Digital Wellness in University Communities: Libraries as Guardians of Healthy...
bySylvester Ebhonu
 
Cost of Capital - Cost of Equity, Cost of debenture, Cost of Preference share...
bySundar B N
 
The Pity of War: Form, Fragment, and the Artificial Echo | Understanding War ...
byNidhi Pandya
 
Orchard Floor Managment.pdf Orchard Floor Management
bybisensharad
 
The Art Pastor's Guide to the Liturgical Calendar
bySteve Thomason
 
Rectal Surgery in Senior Citiizens .pptx
byJohn Thanakumar
 
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
1ST APPLICATION FOR ANNULMENT (4)8787666.pdf
bykonstantinantountoum1
 
Projecte de la porta de la classe de primer A: Mar i cel.
byeduardrubio1
 
The Tale of Melon City poem ppt by Sahasra
bybitrasahasra
 
TAMIS & TEMS - HOW, WHY and THE STEPS IN PROCTOLOGY
byJohn Thanakumar
 
Limpitlaw "Licensing: From Mindset to Milestones"
byNational Information Standards Organization (NISO)
 
Details of Epithelial and Connective Tissue.pptx
byAshish Umale
 
ICH Harmonization A Global Pathway to Unified Drug Regulation.pptx
byDr. Smita Kumbhar
 

Convolutional Neural Network Models - Deep Learning

  • 1.
  • 2.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 3.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 4.
    CNN Models  ConvolutionalNeural Network (CNN)is a multi-layer neural network  Convolutional Neural Network is comprised of one or more convolutional layers (often with a pooling layers) and then followed by one or more fully connected layers.
  • 5.
    CNN Models  Convolutionallayer acts as a feature extractor that extracts features of the inputs such as edges, corners , endpoints.
  • 6.
    CNN Models  Poolinglayer reduces the resolution of the image that reduce the precision of the translation (shift and distortion) effect.
  • 7.
    CNN Models  fullyconnected layer have full connections to all activations in the previous layer.  Fully connect layer act as classifier.
  • 8.
    CNN Models Output Image= ( ( (ImageSize+2*Padding)- KernalSize )/ Stride) +1
  • 9.
    CNN Models  Conv3x3 with stride=1,padding=0 6x6 Image 4x4
  • 10.
    CNN Models  Conv3x3 with stride=1,padding=1 4x4 Image 4x4
  • 11.
    CNN Models  Conv3x3 with stride=2,padding=0 7x7 Image 3x3
  • 12.
    CNN Models  Conv3x3 with stride=2,padding=1 5x5 Image 3x3
  • 13.
    CNN Models  MaxPooling2x2 with stride=2 4x4 Image 2x2
  • 14.
    CNN Models  MaxPooling3x3 with stride=2 7x7 Image 3x3
  • 15.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 16.
    CNN Models  ImageNetLarge Scale Visual Recognition Challenge is image classification challenge to create model that can correctly classify an input image into 1,000 separate object categories. Models are trained on 1.2 million training images with another 50,000 images for validation and 150,000 images for testing
  • 17.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 18.
    CNN Models  AlexNetachieve on ILSVRC 2012 competition 15.3% Top-5 error rate compare to 26.2% achieved by the second best entry.  AlexNet using batch stochastic gradient descent on training, with specific values for momentum and weight decay.  AlexNet implement dropout layers in order to combat the problem of overfitting to the training data.
  • 19.
    CNN Models Image Conv1 Pool1 Conv2 Pool2 Conv3 Conv4 Conv5 Pool3 FC1 FC2 FC3  AlexNethas 8 layers without count pooling layers.  AlexNet use ReLU for the nonlinearity functions  AlexNet trained on two GTX 580 GPUs for five to six days
  • 20.
    CNN Models Image 227x227x3 Conv11-96 MaxpoolConv5-256 MaxpoolConv3-384Conv3-384Conv3-256 Maxpool FC-4096 FC-4096 FC-1000
  • 21.
  • 22.
    CNN Models  Layer0: Input image  Size: 227 x 227 x 3  Memory: 227 x 227 x 3
  • 23.
    CNN Models  Layer0: 227 x 227 x 3  Layer 1: Convolution with 96 filters, size 11×11, stride 4, padding 0  Outcome Size= 55 x 55 x 96  (227-11)/4 + 1 = 55 is size of outcome  Memory: 55 x 55 x 96 x 3 (because of ReLU & LRN(Local Response Normalization))  Weights (parameters) : 11 x 11 x 3 x 96
  • 24.
    CNN Models  Layer1: 55 x 55 x 96  Layer 2: Max-Pooling with 3×3 filter, stride 2  Outcome Size= 27 x 27 x 96  (55 – 3)/2 + 1 = 27 is size of outcome  Memory: 27 x 27 x 96
  • 25.
    CNN Models  Layer2: 27 x 27 x 96  Layer 3: Convolution with 256 filters, size 5×5, stride 1, padding 2  Outcome Size = 27 x 27 x 256  original size is restored because of padding  Memory: 27 x 27 x 256 x 3 (because of ReLU and LRN)  Weights: 5 x 5 x 96 x 256
  • 26.
    CNN Models  Layer3: 27 x 27 x 256  Layer 4: Max-Pooling with 3×3 filter, stride 2  Outcome Size = 13 x 13 x 256  (27 – 3)/2 + 1 = 13 is size of outcome  Memory: 13 x 13 x 256
  • 27.
    CNN Models  Layer4: 13 x 13 x 256  Layer 5: Convolution with 384 filters, size 3×3, stride 1, padding 1  Outcome Size = 13 x 13 x 384  the original size is restored because of padding (13+2 -3)/1 +1 =13  Memory: 13 x 13 x 384 x 2 (because of ReLU)  Weights: 3 x 3 x 256 x 384
  • 28.
    CNN Models  Layer5: 13 x 13 x 384  Layer 6: Convolution with 384 filters, size 3×3, stride 1, padding 1  Outcome Size = 13 x 13 x 384  the original size is restored because of padding  Memory: 13 x 13 x 384 x 2 (because of ReLU)  Weights: 3 x 3 x 384 x 384
  • 29.
    CNN Models  Layer6: 13 x 13 x 384  Layer 7: Convolution with 256 filters, size 3×3, stride 1, padding 1  Outcome Size = 13 x 13 x 256  the original size is restored because of padding  Memory: 13 x 13 x 256 x 2 (because of ReLU)  Weights: 3 x 3 x 384 x 256
  • 30.
    CNN Models  Layer7: 13 x 13 x 256  Layer 8: Max-Pooling with 3×3 filter, stride 2  Outcome Size = 6 x 6 x 256  (13 – 3)/2 + 1 = 6 is size of outcome  Memory: 6 x 6 x 256
  • 31.
    CNN Models  Layer8: 6x6x256=9216 pixels are fed to FC  Layer 9: Fully Connected with 4096 neuron  Memory: 4096 x 3 (because of ReLU and Dropout)  Weights: 4096 x (6 x 6 x 256)
  • 32.
    CNN Models  Layer9: Fully Connected with 4096 neuron  Layer 10: Fully Connected with 4096 neuron  Memory: 4096 x 3 (because of ReLU and Dropout)  Weights: 4096 x 4096
  • 33.
    CNN Models  Layer10: Fully Connected with 4096 neuron  Layer 11: Fully Connected with 1000 neurons  Memory: 1000  Weights: 4096 x 1000
  • 34.
    CNN Models  Total(label and softmax not included)  Memory: 2.24 million  Weights: 62.37 million
  • 35.
    CNN Models  firstuse of ReLU  Alexnet used Norm layers  Alexnet heavy used data augmentation  Alexnet use dropout 0.5  Alexnet batch size is 128  Alexnet used SGD Momentum 0.9  Alexnet used learning rate 1e-2, reduced by 10
  • 36.
    CNN Models [227x227x3] INPUT [55x55x96]CONV1 : 96 11x11 filters at stride 4, pad 0 27x27x96] MAX POOL1 : 3x3 filters at stride 2 [27x27x96] NORM1: Normalization layer [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 [13x13x256] MAX POOL2: 3x3 filters at stride 2 [13x13x256] NORM2: Normalization layer
  • 37.
    CNN Models [13x13x384] CONV3:384 3x3 filters at stride 1, pad 1 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 [6x6x256] MAX POOL3: 3x3 filters at stride 2 [4096] FC6: 4096 neurons [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons
  • 38.
    CNN Models  ImplementAlexNet using TFLearn
  • 39.
  • 40.
  • 41.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 42.
    CNN Models  ZFNetthe winner of the competition ILSVRC 2013 with 14.8% Top-5 error rate  ZFNet built by Matthew Zeiler and Rob Fergus  ZFNet has the same global architecture as Alexnet, that is to say 5 convolutional layers, two fully connected layers and an output softmax one. The differences are for example better sized convolutional kernels.
  • 43.
    CNN Models  ZFNetused filters of size 7x7 and a decreased stride value, instead of using 11x11 sized filters in the first layer (which is what AlexNet implemented).  ZFNet trained on a GTX 580 GPU for twelve days.  Developed a visualization technique named Deconvolutional Network “deconvnet” because it maps features to pixels.
  • 44.
    CNN Models AlexNet but: •CONV1: change from (11x11 stride 4) to (7x7 stride 2) • CONV3,4,5: instead of 384, 384, 256 filters use 512, 1024, 512
  • 45.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 46.
    CNN Models  Keepit deep. Keep it simple.  VGGNet the runner up of the competition ILSVRC 2014 with 7.3% Top-5 error rate.  VGGNet use of only 3x3 sized filters is quite different from AlexNet’s 11x11 filters in the first layer and ZFNet’s 7x7 filters.  two 3x3 conv layers have an effective receptive field of 5x5  Three 3x3 conv layers have an effective receptive field of 7x7  VGGNet trained on 4 Nvidia Titan Black GPUs for two to three weeks
  • 47.
    CNN Models  Interestingto notice that the number of filters doubles after each maxpool layer. This reinforces the idea of shrinking spatial dimensions, but growing depth.  VGGNet used scale jittering as one data augmentation technique during training  VGGNet used ReLU layers after each conv layer and trained with batch gradient descent
  • 48.
  • 49.
    CNN Models Input 224x224x3 Conv3-64 Conv3-64Maxpool Conv3-128 Conv3-128 MaxpoolConv3-256Conv3-256Conv3-256MaxpoolConv3-512 Conv3-512 Conv3-512 Maxpool Conv3-512 Conv3-512 Conv3-512 MaxpoolFC-4096FC-4096FC-1000VGGNet 16
  • 50.
  • 51.
    CNN Models Input 224x224x3 Conv3-64 Conv3-64Maxpool Conv3-128 Conv3-128 Maxpool Conv3-256Conv3-256Conv3-256Conv3-256MaxpoolConv3-512Conv3-512 Conv3-512 Conv3-512 Maxpool Conv3-512 Conv3-512 Conv3-512 Conv3-512 MaxpoolFC-4096FC-4096FC-1000 VGGNet 19
  • 52.
    CNN Models  ImplementVGGNet16 using TFLearn
  • 53.
  • 54.
  • 55.
  • 56.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 57.
    CNN Models  GoogleNetis the winner of the competition ILSVRC 2014 with 6.7% Top-5 error rate.  GoogleNet Trained on “a few high-end GPUs with in a week”  GoogleNet uses 12x fewer parameters than AlexNet  GoogleNet use an average pool instead of fully connected layers, to go from a 7x7x1024 volume to a 1x1x1024 volume. This saves a huge number of parameters.
  • 58.
    CNN Models  GoogleNetused 9 Inception modules in the whole architecture  This 1x1 convolutions (bottleneck convolutions) allow to control/reduce the depth dimension which greatly reduces the number of used parameters due to removal of redundancy of correlated filters.  GoogleNet has 22 Layers deep network
  • 59.
    CNN Models  GoogleNetuse an average pool instead of using FC-Layer, to go from a 7x7x1024 volume to a 1x1x1024 volume. This saves a huge number of parameters.  GoogleNet use inexpensive Conv1 to compute reduction before the expensive Conv3 and Conv5  Conv1 follow by Relu to reduce overfitting
  • 60.
  • 61.
    CNN Models Input 224x224x3 Conv7/2-64 Maxpool3/2Conv1 Conv3/1- 192 Maxpool3/2 Inception3a 256 Inception3b 480 Maxpool3/2 Inception4a 512 Inception4b 512 Inception4c 512 Inception4d 528 Inception4e 832 Maxpool3/2 Inception5a 832 Inception5b 1024 Avgpool7/1 Dropout 40% FC-1000 Softmax- 1000GoogleNet
  • 62.
  • 63.
    CNN Models Type Size/ Stride Output Depth Conv1 # Conv3 Conv3 # Conv5 Conv5 Pool Param Ops Conv 7x7/2112x112x64 1 - - - - - - 2.7K 34M Maxpool 3x3/2 56x56x64 0 - - - - - - - - Conv 3x3/1 56x56x192 2 - 64 192 - - - 112K 360M Maxpool 3x3/2 28x28x192 0 - - - - - - - - Inception 3a - 28x28x256 2 64 96 128 16 32 32 159K 128M Inception 3b - 28x28x480 2 128 128 192 32 96 64 380K 304M Maxpool 3x3/2 14x14x480 0 - - - - - - - - Inception 4a - 14x14x512 2 192 96 208 16 48 64 364K 73M Inception 4b - 14x14x512 2 160 112 224 24 64 64 437K 88M Inception 4c - 14x14x512 2 128 128 256 24 64 64 463K 100M Inception 4d - 14x14x528 2 112 144 288 32 64 64 580K 119M
  • 64.
    CNN Models Type Size/ Stride Output Depth Conv1 # Conv3 Conv3 # Conv5 Conv5 Pool Param Ops Inception 4e- 14x14x832 2 256 160 320 32 128 128 840K 170M Maxpool 3x3/2 7x7x832 0 - - - - - - - - Inception 5a - 7x7x832 2 256 160 320 32 128 128 1072K 54M Inception 5b - 7x7x1024 2 384 192 384 48 128 128 1388K 71M Avgpool 7x7/1 1x1x1024 0 - - - - - - - - Dropout .4 - 1x1x1024 0 - - - - - - - - Linear - 1x1x1024 1 - - - - - - 1000K 1M Softmax - 1x1x1024 0 - - - - - - - - Total Layers 22
  • 65.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 66.
    CNN Models  ResNetthe winner of the competition ILSVRC 2015 with 3.6% Top-5 error rate.  ResNet mainly inspired by the philosophy of VGGNet.  ResNet proposed a residual learning approach to ease the difficulty of training deeper networks. Based on the design ideas of Batch Normalization (BN), small convolutional kernels.  ResNet is a new 152 layer network architecture.  ResNet Trained on an 8 GPU machine for two to three weeks
  • 67.
    CNN Models  Residualnetwork  Keys:  No max pooling  No hidden fc  No dropout  Basic design (VGG-style)  All 3x3 conv (almost)  Batch normalization
  • 68.
    CNN Models Conv Layers Preserving baseinformation can treat perturbation
  • 69.
  • 70.
    CNN Models  ResidualBottleneck consist of a 1×1 layer for reducing dimension, a 3×3 layer, and a 1×1 layer for restoring dimension.
  • 71.
    CNN Models Image Conv7/2- 64 Pool/2 Conv3-64Conv3-64 Conv3-64 Conv3-64 Conv3-64 Conv3-64 Conv3/2- 128 Conv3-128Conv3-128Conv3-128Conv3-128Conv3-128Conv3-128 Conv3-128 Conv3/2- 256 Conv3-256 Conv3-256 Conv3-256 Conv3-256 Conv3-256 Conv3-256 Conv3-256Conv3-256Conv3-256Conv3-256Conv3-256 Conv3/2- 512 Conv3-512Conv3-512 Conv3-512 Conv3-512 Conv3-512 Avg pool FC-1000 ResNet 34
  • 72.
    CNN Models Image Conv7/2-64Pool/2 2Conv3-64 2Conv3-64 2Conv3-64 2Conv3/2- 128 2Conv3-1282Conv3-1282Conv3-128 2Conv3/2- 256 2Conv3-256 2Conv3-256 2Conv3-256 2Conv3-256 2Conv3-256 2Conv3/2- 512 2Conv3-512 2Conv3-512Avg poolFC-1000ResNet 34
  • 73.
  • 74.
    CNN Models Layer Output18-Layer 34-Layer 50-Layer 101-Layer 152-Layer Conv-1 112x112 7x7/2-64 Conv-2 56x56 3x3 Maxpooling/2 𝟐𝐱 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱𝟑, 𝟔𝟒 𝟑𝐱 𝟏𝐱𝟏, 𝟔𝟒 𝟑𝐱𝟑𝐱𝟔𝟒 𝟏𝐱𝟏𝐱𝟐𝟓𝟔 𝟑𝐱 𝟏𝐱𝟏, 𝟔𝟒 𝟑𝐱𝟑𝐱𝟔𝟒 𝟏𝐱𝟏𝐱𝟐𝟓𝟔 𝟑𝐱 𝟏𝐱𝟏, 𝟔𝟒 𝟑𝐱𝟑𝐱𝟔𝟒 𝟏𝐱𝟏𝐱𝟐𝟓𝟔 Conv-3 28x28 𝟐𝐱 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟒𝐱 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟑𝐱𝟑, 𝟏𝟐𝟖 𝟒𝐱 𝟏𝐱𝟏, 𝟏𝟐𝟖 𝟑𝐱𝟑𝐱𝟏𝟐𝟖 𝟏𝐱𝟏𝐱𝟓𝟏𝟐 𝟒𝐱 𝟏𝐱𝟏, 𝟏𝟐𝟖 𝟑𝐱𝟑𝐱𝟏𝟐𝟖 𝟏𝐱𝟏𝐱𝟓𝟏𝟐 𝟖𝐱 𝟏𝐱𝟏, 𝟏𝟐𝟖 𝟑𝐱𝟑𝐱𝟏𝟐𝟖 𝟏𝐱𝟏𝐱𝟓𝟏𝟐 Conv-4 14x14 𝟐𝐱 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟔𝐱 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟑𝐱𝟑, 𝟐𝟓𝟔 𝟔𝐱 𝟏𝐱𝟏, 𝟐𝟓𝟔 𝟑𝐱𝟑𝐱𝟐𝟓𝟔 𝟏𝐱𝟏𝐱𝟏𝟎𝟐𝟒 𝟐𝟑𝐱 𝟏𝐱𝟏, 𝟐𝟓𝟔 𝟑𝐱𝟑𝐱𝟐𝟓𝟔 𝟏𝐱𝟏𝐱𝟏𝟎𝟐𝟒 𝟑𝟔𝐱 𝟏𝐱𝟏, 𝟐𝟓𝟔 𝟑𝐱𝟑𝐱𝟐𝟓𝟔 𝟏𝐱𝟏𝐱𝟏𝟎𝟐𝟒 Conv-5 7x7 𝟐𝐱 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱𝟑, 𝟓𝟏𝟐 𝟑𝐱 𝟏𝐱𝟏, 𝟓𝟏𝟐 𝟑𝐱𝟑𝐱𝟓𝟏𝟐 𝟏𝐱𝟏𝐱𝟐𝟎𝟒𝟖 𝟑𝐱 𝟏𝐱𝟏, 𝟓𝟏𝟐 𝟑𝐱𝟑𝐱𝟓𝟏𝟐 𝟏𝐱𝟏𝐱𝟐𝟎𝟒𝟖 𝟑𝐱 𝟏𝐱𝟏, 𝟓𝟏𝟐 𝟑𝐱𝟑𝐱𝟓𝟏𝟐 𝟏𝐱𝟏𝐱𝟐𝟎𝟒𝟖 1x1 Avgpool-FC1000-Softmax Flops 𝟏. 𝟖𝐱𝟏𝟎 𝟗 𝟑. 𝟔𝐱𝟏𝟎 𝟗 𝟑. 𝟖𝐱𝟏𝟎 𝟗 𝟕. 𝟔𝐱𝟏𝟎 𝟗 𝟏𝟏. 𝟑𝐱𝟏𝟎 𝟗
  • 75.
    CNN Models  ImplementResNet using TFLearn
  • 76.
  • 77.
  • 78.
  • 79.
    CNN Models Convolutional NeuralNetwork ILSVRC AlexNet (2012) ZFNet (2013) VGGNet (2014) GoogleNet 2014) ResNet (2015) Conclusion
  • 80.
    CNN Models 26.2 15.3 14.8 7.36.7 3.6 0 5 10 15 20 25 30 Before 2012 AlexNet 2012 ZFNet 2013 VGGNet 2014 GoogleNet 2014 ResNet 2015
  • 81.
  • 82.
  • 83.
    CNN Models www.YourCompany.com © 2020Companyname PowerPoint Business Theme. All Rights Reserved. THANKS FOR YOUR TIME