CVPR 2015

Open Source Structure-from-Motion

CVPR 2015 Tutorial

June 7, 2015, Boston, MA

Full Day – Room 206

Organizers

• Matt Leotta  (Kitware, Inc., USA): MAP-Tk
• Sameer Agarwal (Google, Inc.): Ceres Solver
• Frank Dellaert (Georgia Tech): GTSAM
• Pierre Moulon (Foxel SA): OpenMVG
• Vincent Rabaud (Aldebaran – Softbank Group): OpenCV

Course Description

This tutorial is a hands-on introduction to using open source software for solving structure-from-motion (SfM) estimation problems from imagery and video. The goal is to teach attendees how to combine existing open source libraries to build complete, customized SfM estimation pipelines. Topics include feature detection, feature matching/tracking, projective geometry, camera calibration, factor graphs, sparse bundle adjustment, etc. and how to combine all of these components for applications in processing video or unordered image collections. Example code for complete processing pipelines will be provided along with sample data, and attendees will complete hands on exercises to modify, build, and run the code. The focus is on C++ libraries that are permissively licensed (BSD, Apache, MPL, etc.) to allow for use in commercial applications. Primary software packages covered are OpenCV, Ceres Solver, GTSAM, OpenMVG, and MAP-Tk.

Attendees will receive a virtual appliance (VirtualBox) with the full software environment pre-configured, and will execute programming exercises ranging from the introductory level to the intermediate level. The course material, including the virtual appliance, will be distributed on USB memory sticks. Attendees wishing to participate should bring a laptop with modern 64-bit operating system (Windows, Mac, Linux) with at least 4 GB of RAM and 8GB of free disk space. Pre-installing the latest Virtualbox will save time, but is not required. Virtualbox installers will also be provided on the USB stick for those who need it.

Syllabus and Schedule

Motivation

Recovery of structure and motion from imagery and video has long been a key research topic in the computer vision community. As these techniques have matured and become scalable to big data problems, the demand for applying these techniques to real-world problems has also grown. Popular applications range from camera tracking for CGI in filmmaking to city-scale reconstruction from large community photo collections to robot visual navigation. Building a software pipeline for one of these applications requires a combination of numerous complex vision algorithms. Rather than start from scratch, it usually makes more sense to build on the open source implementations of these algorithms that have been provided and tested by experts in the community.

In recent years, a number of open source packages have appeared to address different pieces of the SfM pipeline. Some packages, like the popular Bundler software, provide an end-to-end application to address a specific problem. Bundler provides a complete SfM pipeline for unorganized photo collections, but is minimally customizable and not as easily reconfigured to efficiently apply to other applications, like video. Conversely, other software packages provide APIs for more easily accessible building blocks which can be combined in custom ways to build a SfM pipeline. However, expertise is require to combine these components into a functional system.

This tutorial focuses on the latter case. It is a practical guide for building SfM software applications by building on various open source software packages. While we will briefly cover end-to-end open source applications, like Bundler, the focus will be on how to use the APIs of various open source toolkits to build customized solutions. The tutorial will primarily cover usage of components from OpenCV, Ceres, GTSAM, OpenMVG, and MAP-Tk. We will cover the capabilities of each package as they relate to the SfM problem and give examples of how to use the APIs provided.

Target Audience

This tutorial is for students, professors, researchers, and software developers who are interested in developing software to solve structure and motion estimation problems. In particular this tutorial is relevant to those doing commercial work, because all software covered is open source and is permissively licensed so that it can be integrated into commercial software products with no license fees and few restrictions. Attendees are suggested to have some prior C++ programming experience as well as basic familiarity with structure from motion concepts.

Course Materials

Course materials will be distributed during the tutorial on USB memory sticks. The same materials are available for download. Note that the VirtualBox image is a large download (2.7GB). If you don’t wish to run the VM you can view the source code for the exercises on GitHub. The VM image is full reproducible using the Vagrant tool.

Giveaway

We are giving away a Parrot Bebop Drone to one lucky attendee! You can use this drone to collect your own data for running open source structure from motion algorithms. You must be present at the tutorial and participating to win. Fill out our tutorial feedback survey to enter the drawing.

* The presenters and their companies are not endorsing this product and are not responsible the winner’s actions. Please fly safely and adhere to all local regulations.