Recognizing Scene Viewpoint using Panoramic Place Representation
The pose of an object carries crucial semantic meaning
for object manipulation and usage (e.g., grabbing a mug,
watching a television). Just as pose estimation is part of object recognition, viewpoint recognition is a necessary and
unavoidable component of scene recognition. For instance,
as shown in Figure 1, a theater has a clear distinct distributions of objects – a stage on one side and seats on the other
– that deﬁnes unique views in different orientations. Just
as observers will choose a view of a television that allows
them to see the screen, observers in a theater will sit facing
the stage when watching a show.
The goal of this paper is to study the viewpoint recognition problem in scenes. We aim to design a model which,
given a photo, can classify the place category to which it
belongs (e.g. a theater), and predict the direction in which
the observer is facing within that place (e.g. towards the
stage). Our model learns the typical arrangement of visual
features in a 360-degree
panoramic representation of a place, and
learns to map individual views of a place to that representation. Now, given an input photo, we will be able to place
that photo within a larger panoramic image. This allows us
to extrapolate the layout beyond the available view, as if we
were to rotate the camera all around the observer.
CVPR2012 code: This folder contains all source code and data used in the experiments. It contains all precomputed results as well as source code to recompute everything from scratch.
If you just want to do the viewpoint recognition experiment and compare with our paper, you only need to download this file (no need to download the above links for SUN360 database).
pano2photo: This is a small piece of code to demonstrate how to warp between panorama and normal images. It has been included in the above file.
polarPlot: This is a small piece of code to plot a curve or a histogram in polar coordinate. It has been included in the above file.
OnlineStructuralSVM: a Matlab implementation of the cutting plane algorithm for training a Structural SVM.
We thank Tomasz Malisiewicz, Andrew Owens, Aditya
Khosla, Dahua Lin and reviewers for helpful discussions.
This work is funded by NSF grant (1016862) to A.O,
Google research awards to A.O and A.T., ONR MURI
N000141010933 and NSF Career Award No. 0747120 to
A.T., and a NSF Graduate Research fellowship to K.A.E.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation and other funding agencies.
All materials in this website, including images, data, and visualization, can be used for academic research purpose ONLY.