SlideShare a Scribd company logo

Deep image retrieval - learning global representations for image search - ub version

Download as pptx, pdf
Universitat de Barcelona
Universitat de Barcelona

This document summarizes a research paper on deep image retrieval using global image representations. It presents three key ideas: 1) A siamese network trained with a triplet loss to learn image representations optimized for retrieval. 2) Replacing rigid region grids with a region proposal network to localize regions of interest. 3) Experiments showing their method outperforms classification features and achieves state-of-the-art results on standard retrieval datasets. Their work demonstrates an effective and scalable approach to image retrieval based on learning compact global image signatures.

1 of 26
Downloaded 15 times
Deep Image Retrieval: Learning global representations for image search Albert Gordo, Jon Almazan, Jerome Revaud, Diane Larlus Original Slides by Albert Jiménez Computer Vision Reading Group 1 [arXiv]
1.Introduction 2
3 Instance Retrieval + Ranking 1. 2. 3. 4.Image Retrieval Slide credit: Amaia Sal Ranking Image Query
CNN-based retrieval ● CNNs trained for classification tasks ● Features are very robust to intra-class variability ● Lack of robustness to scaling, cropping and image clutter Related Work Lamp We are interested in distinguishing between particular objects from the same class! 4
R-MAC ● Regional Maximum Activation of Convolutions ● Compact feature vectors encode image regions Related Work Giorgos Tolias, Ronan Sicre, Hervé Jégou, Particular object retrieval with integral max-pooling of CNN activations (Submitted to ICLR 2016) 5
R-MAC ● Regions selected using a rigid grid ● Compute a feature vector per region ● Combine all region feature vectors ○ Dimension → 256 / 512 Related Work Giorgos Tolias, Ronan Sicre, Hervé Jégou, Particular object retrieval with integral max-pooling of CNN activations (Submitted to ICLR 2016) ConvNet Last Layer K feature maps size = W x H Different scale region grids maximum activation 6
2. Methodology 7
1st Contribution ● Three-stream siamese network ● PCA implemented as a shift + fully connected layer ● Optimize weights (CNN + PCA) from R-MAC representation with a triplet loss function 8
where: ● m is a scalar that controls the margin ● q, d+, d- are the descriptors for the query, positive and negative images 1st Contribution Ranking Loss Function 9
2nd Contribution ● Localize regions of interest (ROIs) ● Train a Region Proposal Network with bounding boxes (Similar Fast R-CNN, [arXiv]) In R-MAC → Rigid grid Replace Region Proposal Network 10
2nd Contribution RPN in a nutshell 11 ● Predict, for a set of candidate boxes of various sizes and aspects ratio, and at all possible image locations, a score describing how likely each box contains an object of interest. ● Simultaneously, for each candidate box perform regression to improve its location.
Summary 12 ● Able to encode one image into a compact feature vector in a single forward pass ● Images can be compared using the dot product ● Very efficient at test time
3. Experiments 13
Datasets 14 ● Training Landmarks dataset: 214k images from 672 landmark sites ● Testing Oxford 5k, Paris 6k, Oxford 105k, Paris 106k, INRIA Holidays ● Remove all images contained in Oxford 5k and Paris 6k datasets ○ Landmarks-full: 200k images from 592 landmarks ● Cleaning Landmarks dataset (Select most relevant images/discard incorrect) ○ SIFT + Hessian Affine keypoint det. → Construct graph of similar images ○ Landmarks-clean: 52k images from 592 landmarks
Bounding Box Estimation 15 ● RPN trained using automatically estimated bounding box annotations 1. Define initial bounding box: min rectangle that encloses all matched keypoints 2. For a pair (i, j) we predict the bounding box Bj using Bi and an affine transform Aij 3. Update (Merge using geometrical mean) 4. Iterate until convergence Bounding box projections Initial vs Final estimations
Experimental Details 16 ● VGG-16 network pre-trained on ImageNet ● Fine-tune with Landmarks dataset ● Select triplets in an efficient manner ○ Forward pass to obtain image representations ○ Select hard negatives (Large loss) ● Dimension of the feature vector = 512 ● Evaluation: mean Average Precision (mAP) VGG16
1st Experiment 17 Comparison between R-MAC and their implementations C: Classification Network R: Ranking (Trained with triplets)
2nd Experiment 18 Comparison between fixed grid vs number of region proposals 16-32 proposals already outperform rigid grid!
2nd Experiment 19 mAP - Number of triplets Recall - Number of region proposals
2nd Experiment 20 Heatmap vs Bounding Box Estimation
Comparison with state of the art 21
Comparison with state of the art 22
Top Retrieval Results 23
4. Conclusions 24
Conclusions 25 ● They have proposed an effective and scalable method for image retrieval that encodes images into compact global signatures that can be compared with the dot-product. ● Proposal of a siamese network architecture trained for the specific task of image retrieval using ranking loss function (Triplets). ● Demonstrate the benefit of predicting the ROI of the images when encoding by using Region Proposal Networks.
Thank You! Questions? 26
Ad

Recommended

Convolutional Patch Representations for Image Retrieval An unsupervised approach
Convolutional Patch Representations for Image Retrieval An unsupervised approachConvolutional Patch Representations for Image Retrieval An unsupervised approach
Convolutional Patch Representations for Image Retrieval An unsupervised approach
Universitat de Barcelona
 
Deep image retrieval learning global representations for image search
Deep image retrieval learning global representations for image searchDeep image retrieval learning global representations for image search
Deep image retrieval learning global representations for image search
Universitat Politècnica de Catalunya
 
Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval
Universitat Politècnica de Catalunya
 
Convolutional Features for Instance Search
Convolutional Features for Instance SearchConvolutional Features for Instance Search
Convolutional Features for Instance Search
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Universitat Politècnica de Catalunya
 
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Object Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning FrameworkObject Detection Using R-CNN Deep Learning Framework
Object Detection Using R-CNN Deep Learning Framework
Nader Karimi
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
Dat Nguyen
 
Object detection - RCNNs vs Retinanet
Object detection - RCNNs vs RetinanetObject detection - RCNNs vs Retinanet
Object detection - RCNNs vs Retinanet
Rishabh Indoria
 
object detection paper review
object detection paper reviewobject detection paper review
object detection paper review
Yoonho Na
 
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)
Hwa Pyung Kim
 
Semantic segmentation with Convolutional Neural Network Approaches
Semantic segmentation with Convolutional Neural Network ApproachesSemantic segmentation with Convolutional Neural Network Approaches
Semantic segmentation with Convolutional Neural Network Approaches
UMBC
 
Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
Universitat Politècnica de Catalunya
 
Fast Non-Uniform Filtering with Symmetric Weighted Integral Images
Fast Non-Uniform Filtering with Symmetric Weighted Integral ImagesFast Non-Uniform Filtering with Symmetric Weighted Integral Images
Fast Non-Uniform Filtering with Symmetric Weighted Integral Images
davidmarimon
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite Imagery
RAHUL BHOJWANI
 
Deep learning based object detection basics
Deep learning based object detection basicsDeep learning based object detection basics
Deep learning based object detection basics
Brodmann17
 
R-FCN : object detection via region-based fully convolutional networks
R-FCN : object detection via region-based fully convolutional networksR-FCN : object detection via region-based fully convolutional networks
R-FCN : object detection via region-based fully convolutional networks
Entrepreneur / Startup
 
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Universitat Politècnica de Catalunya
 
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
 
DNR - Auto deep lab paper review ppt
DNR - Auto deep lab paper review pptDNR - Auto deep lab paper review ppt
DNR - Auto deep lab paper review ppt
taeseon ryu
 
Semantic Mapping of Road Scenes
Semantic Mapping of Road ScenesSemantic Mapping of Road Scenes
Semantic Mapping of Road Scenes
Sunando Sengupta
 
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Segmentation (UPC 2016)
Deep Learning for Computer Vision: Segmentation (UPC 2016)Deep Learning for Computer Vision: Segmentation (UPC 2016)
Deep Learning for Computer Vision: Segmentation (UPC 2016)
Universitat Politècnica de Catalunya
 
Shai Avidan's Support vector tracking and ensemble tracking
Shai Avidan's Support vector tracking and ensemble trackingShai Avidan's Support vector tracking and ensemble tracking
Shai Avidan's Support vector tracking and ensemble tracking
wolf
 
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
thanhdowork
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
CHENHuiMei
 
Ad

More Related Content

What's hot (20)

Object detection - RCNNs vs Retinanet
Object detection - RCNNs vs RetinanetObject detection - RCNNs vs Retinanet
Object detection - RCNNs vs Retinanet
Rishabh Indoria
 
object detection paper review
object detection paper reviewobject detection paper review
object detection paper review
Yoonho Na
 
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)
Hwa Pyung Kim
 
Semantic segmentation with Convolutional Neural Network Approaches
Semantic segmentation with Convolutional Neural Network ApproachesSemantic segmentation with Convolutional Neural Network Approaches
Semantic segmentation with Convolutional Neural Network Approaches
UMBC
 
Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
Universitat Politècnica de Catalunya
 
Fast Non-Uniform Filtering with Symmetric Weighted Integral Images
Fast Non-Uniform Filtering with Symmetric Weighted Integral ImagesFast Non-Uniform Filtering with Symmetric Weighted Integral Images
Fast Non-Uniform Filtering with Symmetric Weighted Integral Images
davidmarimon
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite Imagery
RAHUL BHOJWANI
 
Deep learning based object detection basics
Deep learning based object detection basicsDeep learning based object detection basics
Deep learning based object detection basics
Brodmann17
 
R-FCN : object detection via region-based fully convolutional networks
R-FCN : object detection via region-based fully convolutional networksR-FCN : object detection via region-based fully convolutional networks
R-FCN : object detection via region-based fully convolutional networks
Entrepreneur / Startup
 
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Universitat Politècnica de Catalunya
 
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
 
DNR - Auto deep lab paper review ppt
DNR - Auto deep lab paper review pptDNR - Auto deep lab paper review ppt
DNR - Auto deep lab paper review ppt
taeseon ryu
 
Semantic Mapping of Road Scenes
Semantic Mapping of Road ScenesSemantic Mapping of Road Scenes
Semantic Mapping of Road Scenes
Sunando Sengupta
 
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Segmentation (UPC 2016)
Deep Learning for Computer Vision: Segmentation (UPC 2016)Deep Learning for Computer Vision: Segmentation (UPC 2016)
Deep Learning for Computer Vision: Segmentation (UPC 2016)
Universitat Politècnica de Catalunya
 
Shai Avidan's Support vector tracking and ensemble tracking
Shai Avidan's Support vector tracking and ensemble trackingShai Avidan's Support vector tracking and ensemble tracking
Shai Avidan's Support vector tracking and ensemble tracking
wolf
 
Object detection - RCNNs vs Retinanet
Object detection - RCNNs vs RetinanetObject detection - RCNNs vs Retinanet
Object detection - RCNNs vs Retinanet
Rishabh Indoria
 
object detection paper review
object detection paper reviewobject detection paper review
object detection paper review
Yoonho Na
 
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Image Segmentation (D3L1 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNN
Chanuk Lim
 
Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)Tutorial on Object Detection (Faster R-CNN)
Tutorial on Object Detection (Faster R-CNN)
Hwa Pyung Kim
 
Semantic segmentation with Convolutional Neural Network Approaches
Semantic segmentation with Convolutional Neural Network ApproachesSemantic segmentation with Convolutional Neural Network Approaches
Semantic segmentation with Convolutional Neural Network Approaches
UMBC
 
Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
Universitat Politècnica de Catalunya
 
Fast Non-Uniform Filtering with Symmetric Weighted Integral Images
Fast Non-Uniform Filtering with Symmetric Weighted Integral ImagesFast Non-Uniform Filtering with Symmetric Weighted Integral Images
Fast Non-Uniform Filtering with Symmetric Weighted Integral Images
davidmarimon
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite Imagery
RAHUL BHOJWANI
 
Deep learning based object detection basics
Deep learning based object detection basicsDeep learning based object detection basics
Deep learning based object detection basics
Brodmann17
 
R-FCN : object detection via region-based fully convolutional networks
R-FCN : object detection via region-based fully convolutional networksR-FCN : object detection via region-based fully convolutional networks
R-FCN : object detection via region-based fully convolutional networks
Entrepreneur / Startup
 
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Universitat Politècnica de Catalunya
 
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Visualization of Deep Learning Models (D1L6 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
 
DNR - Auto deep lab paper review ppt
DNR - Auto deep lab paper review pptDNR - Auto deep lab paper review ppt
DNR - Auto deep lab paper review ppt
taeseon ryu
 
Semantic Mapping of Road Scenes
Semantic Mapping of Road ScenesSemantic Mapping of Road Scenes
Semantic Mapping of Road Scenes
Sunando Sengupta
 
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Object Segmentation (D2L7 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Segmentation (UPC 2016)
Deep Learning for Computer Vision: Segmentation (UPC 2016)Deep Learning for Computer Vision: Segmentation (UPC 2016)
Deep Learning for Computer Vision: Segmentation (UPC 2016)
Universitat Politècnica de Catalunya
 
Shai Avidan's Support vector tracking and ensemble tracking
Shai Avidan's Support vector tracking and ensemble trackingShai Avidan's Support vector tracking and ensemble tracking
Shai Avidan's Support vector tracking and ensemble tracking
wolf
 

Similar to Deep image retrieval - learning global representations for image search - ub version (20)

[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
thanhdowork
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
CHENHuiMei
 
You only look once (YOLO) : unified real time object detection
You only look once (YOLO) : unified real time object detectionYou only look once (YOLO) : unified real time object detection
You only look once (YOLO) : unified real time object detection
Entrepreneur / Startup
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
Edge AI and Vision Alliance
 
“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...
“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...
“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...
Edge AI and Vision Alliance
 
위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등
DACON AI 데이콘
 
Week5-Faster R-CNN.pptx
Week5-Faster R-CNN.pptxWeek5-Faster R-CNN.pptx
Week5-Faster R-CNN.pptx
fahmi324663
 
Faster R-CNN - PR012
Faster R-CNN - PR012Faster R-CNN - PR012
Faster R-CNN - PR012
Jinwon Lee
 
TransNeRF
TransNeRFTransNeRF
TransNeRF
NavneetPaul2
 
Anchor free object detection by deep learning
Anchor free object detection by deep learningAnchor free object detection by deep learning
Anchor free object detection by deep learning
Yu Huang
 
Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331
Jihong Kang
 
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
thanhdowork
 
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
thanhdowork
 
NetVLAD: CNN architecture for weakly supervised place recognition
NetVLAD: CNN architecture for weakly supervised place recognitionNetVLAD: CNN architecture for weakly supervised place recognition
NetVLAD: CNN architecture for weakly supervised place recognition
Geunhee Cho
 
[NS][Lab_Seminar_240819]Re_PolyWorld.pptx
[NS][Lab_Seminar_240819]Re_PolyWorld.pptx[NS][Lab_Seminar_240819]Re_PolyWorld.pptx
[NS][Lab_Seminar_240819]Re_PolyWorld.pptx
thanhdowork
 
Automatic Detection of Window Regions in Indoor Point Clouds Using R-CNN
Automatic Detection of Window Regions in Indoor Point Clouds Using R-CNNAutomatic Detection of Window Regions in Indoor Point Clouds Using R-CNN
Automatic Detection of Window Regions in Indoor Point Clouds Using R-CNN
Zihao(Gerald) Zhang
 
Transformer in Vision
Transformer in VisionTransformer in Vision
Transformer in Vision
Sangmin Woo
 
[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...
[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...
[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...
thanhdowork
 
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
taeseon ryu
 
[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...
[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...
[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...
thanhdowork
 
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
[NS][Lab_Seminar_241118]Relation Matters: Foreground-aware Graph-based Relati...
thanhdowork
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
CHENHuiMei
 
You only look once (YOLO) : unified real time object detection
You only look once (YOLO) : unified real time object detectionYou only look once (YOLO) : unified real time object detection
You only look once (YOLO) : unified real time object detection
Entrepreneur / Startup
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
Edge AI and Vision Alliance
 
“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...
“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...
“Understanding DNN-Based Object Detectors,” a Presentation from Au-Zone Techn...
Edge AI and Vision Alliance
 
위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등위성이미지 객체 검출 대회 - 2등
위성이미지 객체 검출 대회 - 2등
DACON AI 데이콘
 
Week5-Faster R-CNN.pptx
Week5-Faster R-CNN.pptxWeek5-Faster R-CNN.pptx
Week5-Faster R-CNN.pptx
fahmi324663
 
Faster R-CNN - PR012
Faster R-CNN - PR012Faster R-CNN - PR012
Faster R-CNN - PR012
Jinwon Lee
 
TransNeRF
TransNeRFTransNeRF
TransNeRF
NavneetPaul2
 
Anchor free object detection by deep learning
Anchor free object detection by deep learningAnchor free object detection by deep learning
Anchor free object detection by deep learning
Yu Huang
 
Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331Recent Progress on Object Detection_20170331
Recent Progress on Object Detection_20170331
Jihong Kang
 
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
thanhdowork
 
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
[NS][Lab_Seminar_250203]KAG-prompt (1).pptx
thanhdowork
 
NetVLAD: CNN architecture for weakly supervised place recognition
NetVLAD: CNN architecture for weakly supervised place recognitionNetVLAD: CNN architecture for weakly supervised place recognition
NetVLAD: CNN architecture for weakly supervised place recognition
Geunhee Cho
 
[NS][Lab_Seminar_240819]Re_PolyWorld.pptx
[NS][Lab_Seminar_240819]Re_PolyWorld.pptx[NS][Lab_Seminar_240819]Re_PolyWorld.pptx
[NS][Lab_Seminar_240819]Re_PolyWorld.pptx
thanhdowork
 
Automatic Detection of Window Regions in Indoor Point Clouds Using R-CNN
Automatic Detection of Window Regions in Indoor Point Clouds Using R-CNNAutomatic Detection of Window Regions in Indoor Point Clouds Using R-CNN
Automatic Detection of Window Regions in Indoor Point Clouds Using R-CNN
Zihao(Gerald) Zhang
 
Transformer in Vision
Transformer in VisionTransformer in Vision
Transformer in Vision
Sangmin Woo
 
[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...
[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...
[NS][Lab_Seminar_241007]GKGNet: Group K-Nearest Neighbor based Graph Convolut...
thanhdowork
 
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
taeseon ryu
 
[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...
[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...
[NS][Lab_Seminar_240930]ACGT-Net: Adaptive Cuckoo Refinement-Based Graph Tran...
thanhdowork
 
Ad

Recently uploaded (20)

22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
Guru Nanak Technical Institutions
 
Working with USDOT UTCs: From Conception to Implementation
Working with USDOT UTCs: From Conception to ImplementationWorking with USDOT UTCs: From Conception to Implementation
Working with USDOT UTCs: From Conception to Implementation
Alabama Transportation Assistance Program
 
Construction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil EngineeringConstruction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil Engineering
Lavish Kashyap
 
Lecture - 7 Canals of the topic of the civil engineering
Lecture - 7 Canals of the topic of the civil engineeringLecture - 7 Canals of the topic of the civil engineering
Lecture - 7 Canals of the topic of the civil engineering
MJawadkhan1
 
Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...
Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...
Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...
AI Publications
 
sss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptx
sss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptx
sss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptx
ajayrm685
 
acid base ppt and their specific application in food
acid base ppt and their specific application in foodacid base ppt and their specific application in food
acid base ppt and their specific application in food
Fatehatun Noor
 
introduction technology technology tec.pptx
introduction technology technology tec.pptxintroduction technology technology tec.pptx
introduction technology technology tec.pptx
Iftikhar70
 
DED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedungDED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedung
nabilarizqifadhilah1
 
twin tower attack 2001 new york city
twin tower attack 2001 new york citytwin tower attack 2001 new york city
twin tower attack 2001 new york city
harishreemavs
 
Nanometer Metal-Organic-Framework Literature Comparison
Nanometer Metal-Organic-Framework Literature ComparisonNanometer Metal-Organic-Framework Literature Comparison
Nanometer Metal-Organic-Framework Literature Comparison
Chris Harding
 
Artificial intelligence and machine learning.pptx
Artificial intelligence and machine learning.pptxArtificial intelligence and machine learning.pptx
Artificial intelligence and machine learning.pptx
rakshanatarajan005
 
Automatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and BeyondAutomatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and Beyond
NU_I_TODALAB
 
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdfDavid Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry
 
Modeling the Influence of Environmental Factors on Concrete Evaporation Rate
Modeling the Influence of Environmental Factors on Concrete Evaporation RateModeling the Influence of Environmental Factors on Concrete Evaporation Rate
Modeling the Influence of Environmental Factors on Concrete Evaporation Rate
Journal of Soft Computing in Civil Engineering
 
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
PawachMetharattanara
 
Machine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATIONMachine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATION
DarrinBright1
 
Generative AI & Large Language Models Agents
Generative AI & Large Language Models AgentsGenerative AI & Large Language Models Agents
Generative AI & Large Language Models Agents
aasgharbee22seecs
 
Control Methods of Noise Pollutions.pptx
Control Methods of Noise Pollutions.pptxControl Methods of Noise Pollutions.pptx
Control Methods of Noise Pollutions.pptx
vvsasane
 
Frontend Architecture Diagram/Guide For Frontend Engineers
Frontend Architecture Diagram/Guide For Frontend EngineersFrontend Architecture Diagram/Guide For Frontend Engineers
Frontend Architecture Diagram/Guide For Frontend Engineers
Michael Hertzberg
 
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
22PCOAM16 ML Unit 3 Full notes PDF & QB.pdf
Guru Nanak Technical Institutions
 
Working with USDOT UTCs: From Conception to Implementation
Working with USDOT UTCs: From Conception to ImplementationWorking with USDOT UTCs: From Conception to Implementation
Working with USDOT UTCs: From Conception to Implementation
Alabama Transportation Assistance Program
 
Construction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil EngineeringConstruction Materials (Paints) in Civil Engineering
Construction Materials (Paints) in Civil Engineering
Lavish Kashyap
 
Lecture - 7 Canals of the topic of the civil engineering
Lecture - 7 Canals of the topic of the civil engineeringLecture - 7 Canals of the topic of the civil engineering
Lecture - 7 Canals of the topic of the civil engineering
MJawadkhan1
 
Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...
Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...
Empowering Electric Vehicle Charging Infrastructure with Renewable Energy Int...
AI Publications
 
sss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptx
sss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptx
sss1.pptxsss1.pptxsss1.pptxsss1.pptxsss1.pptx
ajayrm685
 
acid base ppt and their specific application in food
acid base ppt and their specific application in foodacid base ppt and their specific application in food
acid base ppt and their specific application in food
Fatehatun Noor
 
introduction technology technology tec.pptx
introduction technology technology tec.pptxintroduction technology technology tec.pptx
introduction technology technology tec.pptx
Iftikhar70
 
DED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedungDED KOMINFO detail engginering design gedung
DED KOMINFO detail engginering design gedung
nabilarizqifadhilah1
 
twin tower attack 2001 new york city
twin tower attack 2001 new york citytwin tower attack 2001 new york city
twin tower attack 2001 new york city
harishreemavs
 
Nanometer Metal-Organic-Framework Literature Comparison
Nanometer Metal-Organic-Framework Literature ComparisonNanometer Metal-Organic-Framework Literature Comparison
Nanometer Metal-Organic-Framework Literature Comparison
Chris Harding
 
Artificial intelligence and machine learning.pptx
Artificial intelligence and machine learning.pptxArtificial intelligence and machine learning.pptx
Artificial intelligence and machine learning.pptx
rakshanatarajan005
 
Automatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and BeyondAutomatic Quality Assessment for Speech and Beyond
Automatic Quality Assessment for Speech and Beyond
NU_I_TODALAB
 
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdfDavid Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry - Specializes In AWS, Microservices And Python.pdf
David Boutry
 
Modeling the Influence of Environmental Factors on Concrete Evaporation Rate
Modeling the Influence of Environmental Factors on Concrete Evaporation RateModeling the Influence of Environmental Factors on Concrete Evaporation Rate
Modeling the Influence of Environmental Factors on Concrete Evaporation Rate
Journal of Soft Computing in Civil Engineering
 
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
01.คุณลักษณะเฉพาะของอุปกรณ์_pagenumber.pdf
PawachMetharattanara
 
Machine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATIONMachine Learning basics POWERPOINT PRESENETATION
Machine Learning basics POWERPOINT PRESENETATION
DarrinBright1
 
Generative AI & Large Language Models Agents
Generative AI & Large Language Models AgentsGenerative AI & Large Language Models Agents
Generative AI & Large Language Models Agents
aasgharbee22seecs
 
Control Methods of Noise Pollutions.pptx
Control Methods of Noise Pollutions.pptxControl Methods of Noise Pollutions.pptx
Control Methods of Noise Pollutions.pptx
vvsasane
 
Frontend Architecture Diagram/Guide For Frontend Engineers
Frontend Architecture Diagram/Guide For Frontend EngineersFrontend Architecture Diagram/Guide For Frontend Engineers
Frontend Architecture Diagram/Guide For Frontend Engineers
Michael Hertzberg
 
Ad

Deep image retrieval - learning global representations for image search - ub version

  • 1. Deep Image Retrieval: Learning global representations for image search Albert Gordo, Jon Almazan, Jerome Revaud, Diane Larlus Original Slides by Albert Jiménez Computer Vision Reading Group 1 [arXiv]
  • 3. 3 Instance Retrieval + Ranking 1. 2. 3. 4.Image Retrieval Slide credit: Amaia Sal Ranking Image Query
  • 4. CNN-based retrieval ● CNNs trained for classification tasks ● Features are very robust to intra-class variability ● Lack of robustness to scaling, cropping and image clutter Related Work Lamp We are interested in distinguishing between particular objects from the same class! 4
  • 5. R-MAC ● Regional Maximum Activation of Convolutions ● Compact feature vectors encode image regions Related Work Giorgos Tolias, Ronan Sicre, Hervé Jégou, Particular object retrieval with integral max-pooling of CNN activations (Submitted to ICLR 2016) 5
  • 6. R-MAC ● Regions selected using a rigid grid ● Compute a feature vector per region ● Combine all region feature vectors ○ Dimension → 256 / 512 Related Work Giorgos Tolias, Ronan Sicre, Hervé Jégou, Particular object retrieval with integral max-pooling of CNN activations (Submitted to ICLR 2016) ConvNet Last Layer K feature maps size = W x H Different scale region grids maximum activation 6
  • 8. 1st Contribution ● Three-stream siamese network ● PCA implemented as a shift + fully connected layer ● Optimize weights (CNN + PCA) from R-MAC representation with a triplet loss function 8
  • 9. where: ● m is a scalar that controls the margin ● q, d+, d- are the descriptors for the query, positive and negative images 1st Contribution Ranking Loss Function 9
  • 10. 2nd Contribution ● Localize regions of interest (ROIs) ● Train a Region Proposal Network with bounding boxes (Similar Fast R-CNN, [arXiv]) In R-MAC → Rigid grid Replace Region Proposal Network 10
  • 11. 2nd Contribution RPN in a nutshell 11 ● Predict, for a set of candidate boxes of various sizes and aspects ratio, and at all possible image locations, a score describing how likely each box contains an object of interest. ● Simultaneously, for each candidate box perform regression to improve its location.
  • 12. Summary 12 ● Able to encode one image into a compact feature vector in a single forward pass ● Images can be compared using the dot product ● Very efficient at test time
  • 14. Datasets 14 ● Training Landmarks dataset: 214k images from 672 landmark sites ● Testing Oxford 5k, Paris 6k, Oxford 105k, Paris 106k, INRIA Holidays ● Remove all images contained in Oxford 5k and Paris 6k datasets ○ Landmarks-full: 200k images from 592 landmarks ● Cleaning Landmarks dataset (Select most relevant images/discard incorrect) ○ SIFT + Hessian Affine keypoint det. → Construct graph of similar images ○ Landmarks-clean: 52k images from 592 landmarks
  • 15. Bounding Box Estimation 15 ● RPN trained using automatically estimated bounding box annotations 1. Define initial bounding box: min rectangle that encloses all matched keypoints 2. For a pair (i, j) we predict the bounding box Bj using Bi and an affine transform Aij 3. Update (Merge using geometrical mean) 4. Iterate until convergence Bounding box projections Initial vs Final estimations
  • 16. Experimental Details 16 ● VGG-16 network pre-trained on ImageNet ● Fine-tune with Landmarks dataset ● Select triplets in an efficient manner ○ Forward pass to obtain image representations ○ Select hard negatives (Large loss) ● Dimension of the feature vector = 512 ● Evaluation: mean Average Precision (mAP) VGG16
  • 17. 1st Experiment 17 Comparison between R-MAC and their implementations C: Classification Network R: Ranking (Trained with triplets)
  • 18. 2nd Experiment 18 Comparison between fixed grid vs number of region proposals 16-32 proposals already outperform rigid grid!
  • 19. 2nd Experiment 19 mAP - Number of triplets Recall - Number of region proposals
  • 20. 2nd Experiment 20 Heatmap vs Bounding Box Estimation
  • 21. Comparison with state of the art 21
  • 22. Comparison with state of the art 22
  • 25. Conclusions 25 ● They have proposed an effective and scalable method for image retrieval that encodes images into compact global signatures that can be compared with the dot-product. ● Proposal of a siamese network architecture trained for the specific task of image retrieval using ranking loss function (Triplets). ● Demonstrate the benefit of predicting the ROI of the images when encoding by using Region Proposal Networks.

Editor's Notes

  • #6: It aggregates several image regions into a compact feature vector of fixed length and is thus robust to scale and translation. This representation can deal with high resolution images of different aspect ratios and obtains a very competitive accuracy.
  • #7: Approximate integral max-pooling.
  翻译：