Download as PDF, PPTX
More Related Content
Similar to Image segmentation with deep learning
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Image segmentation with deep learning
- 1. Image Segmentation with DeepLearning Antonio Rueda-Toicen and Imran Kocabiyik Berlin Computer Vision Group December 2020
- 2.
- 3. Agenda ● Image segmentation ■Semantic segmentation ● Fully convolutional networks, U-net ■ Instance segmentation ● Mask R-CNN ■ Panoptic segmentation ● Feature Pyramid Networks ○ Public datasets ■ COCO ■ Google Open Images ○ Implementations: Detectron2, Fast.ai
- 4. Classification, detection, andsegmentation Classification refers to image-wide labels Detection refers to localization of bounding boxes with labels Segmentation refers to pixel-wise localization of the labels
- 5. Goals of supervisedimage segmentation Given an input image we wish to obtain: 1. A class label associated to each individual pixel in the image. This is also called pixel-wise localization. 3. The probability score associated with each class label
- 6. Applications of imagesegmentation Link
- 7. Applications of imagesegmentation http://withoutbg.com/
- 8. Applications of imagesegmentation
- 9. Applications of imagesegmentation
- 10. Applications of imagesegmentation
- 11. Applications of imagesegmentation https://www.segmentive.ai/
- 12. Segmentation as pixel-wiselocalization
- 13. Instance segmentation requiresobject detection
- 14. Panoptic segmentation https://arxiv.org/pdf/1801.00868.pdf Explore itin the detectron2 inference notebook
- 15. “Fully Convolutional” networksdraw segmentation masks All layers in the network are convolutional, there is no fully connected (aka “dense”) layer like in most classifiers, we use the local info of the pixel neighborhood
- 16. What is aconvolution filter? https://setosa.io/ev/image-kernels/
- 17. What is aconvolution filter? https://setosa.io/ev/image-kernels/
- 18. What is aconvolution filter? Convolution of 3x3 and stride = 1 without padding Effect: the output loses one pixel on each dimension
- 19. What is aconvolution filter? Convolution of 3x3 and stride = 1 with zero padding Effect: the output preserves original image size
- 20. What is aconvolution filter? Convolution of 3x3 and stride = 2 with zero padding Effect: the output is downsampled to about half its size
- 21. “Fully Convolutional” networksdraw segmentation masks All layers in the network are convolutional, there is no fully connected (aka “dense”) layer like in most classifiers, we use the local info of the pixel neighborhood
- 22. U-net for semanticsegmentation All layers in the network are convolutional, there is no fully connected (aka “dense”) layer like in most classifiers, we need this fully convolutional architecture to label images pixel by pixel preserving their local info
- 23. U-net for semanticsegmentation All layers in the network are convolutional, there is no fully connected (aka “dense”) layer like in most classifiers, we need this fully convolutional architecture to label images pixel by pixel preserving their local info
- 24.
- 25. Image Pyramids inFeature Proposal Networks (FPNs)
- 26. Convolutional networks implement“pyramids” The deeper we go into the network, the more semantic value is compressed in lower x,y dimensions
- 27.
- 28. Nearest neighbor interpolation Resnetsin feature pyramid networks
- 29. 1x1 convolution Resnets infeature pyramid networks
- 30.
- 31. Image Pyramids inFeature Proposal Networks (FPNs)
- 32.
- 33.
- 34. The Google OpenImages Dataset
- 35.
- 37.
- 38.
- 39.
- 40. Detectron2 detectron2/MODEL_ZOO.md at master· facebookresearch/detectron2 · GitHub Inference (Colab notebook) Training (Colab notebook) Generating validation set plots
- 41. Panoptic segmentation withfeature pyramid network (FPN-50)
- 42.
- 43.
- 44.
- 45.
- 46. ⊕ Using a Unet ExampleCase: Image Matting
- 47. Matting algorithm: Example Case:Image Matting
- 48. instance segment Using trimapor instance segments? Example Case: Image Matting ⊕ or ⊕ ?
- 49. Results Example Case: ImageMatting Photo: Ayo Ogunseinde https://unsplash.com/photos/THIs-cpyebg
- 50. Results Example Case: ImageMatting Photo: Eugen Proskouriakov https://unsplash.com/photos/C-gvAA8q3Tc
- 51. Results Example Case: ImageMatting Photo: Mathieu Renier https://unsplash.com/photos/4WBvCqeMaDE
- 52. Results Example Case: ImageMatting Photo: Gulyás Bianka https://unsplash.com/photos/3WOh54znPGU
- 53. For more examples: withoutbg.com ExampleCase: Image Matting
- 54. Which things shouldbe kept in this picture? Kid, ball, 2 dogs, 9 people? Example Case: Image Matting Photo: Treddy Chen https://unsplash.com/photos/UdQWvefOXJk
- 55. Issue: When thereis more than one person in the image... Example Case: Image Matting
- 56. Review questions - Howdo we compute the confusion matrix for a segmentation mask? How do we compute it for a bounding box? - Can we use the Intersection over Union equation to evaluate the quality of a segmentation mask? - What’s the recall of a classifier that only outputs ‘1’ (positive class)? - What’s the precision of a classifier that outputs a single true positive, with all its other predictions being equal to ‘0’ (negative class)? - Why does precision go down when recall increases? - Does the F1 measure weigh precision and recall equally? - What’s the appeal of using Detectron2? Do we need to write a Pytorch model to use it for inference or training?
- 57. Google Colab Notebooks ●Unet in FastAI 2 ● Mask R-CNN and Panoptic Segmentation with Detectron 2
- 58. - How doespanoptic segmentation combine instance and semantic segmentation? Which method produces the ‘stuff’? Which method produces the ‘things’? - Is semantic segmentation more computationally costly than instance segmentation? Why? - Is panoptic segmentation more computationally costly than instance segmentation? Why? Review questions
- 59. References ● Stanford’s cs231nlecture on Object Detection and Segmentation ● PyImageSearch tutorial on Mask R-CNN