Table of Contents:

Deadline

11:59PM, Monday, March 18, 2019

Introduction

The aim of this project is to implement an end-to-end pipeline for panorama stitching. We all use the panorama mode on our smart-phones– you’ll implement a pipeline which does the same basic thing!

This document just provides an overview of what you need to do. For a full breakdown of how each step in the pipeline works, see the course notes for this project.

System Overview

Here’s a system diagram, showing each step in your panorama-stitching pipeline:

A brief description of each step (you’ll implement the steps in bold):

  • Cylinder Projection: Project images onto a cylinder, to reduce distortion at the panorama’s edges. For this project, it’s optional.
  • Detect Corners: identify corner points in your images. (You can just use Matlab’s cornermetric.)
  • ANMS: pick out the stronger corner points.
  • Feature Descriptors: create descriptors for the corner points, so they can be matched between images (in the next step).
  • Feature Matching: Match feature descriptors from different images, to find possible point correspondences.
  • RANSAC and Homography Estimation: refine the feature point matches, and use the correspondences to estimate homographies between images.
  • Image Warping (and Blending): Use the estimated homographies to warp the images onto one another, and apply blending to reduce the appearance of seams where they fit together.
    • For blending in this project, you can simply average the pixel values of overlapping images.

Point Distribution:

  • ANMS: 25 pts
  • Feature Descriptors: 15pts
  • Feature Matching: 15pts
  • RANSAC and Homography Estimation: 20pts
  • Image Warping (and Blending): 25pts

Project Files and Starter Code

Please find the starter code at this link.

This also includes three sets of “training images”. Twenty-four hours before the due date, we’ll distribute a “test set” of two more sets of images (look out for an announcement on Piazza).

When writing your program, you can assume that input images will always follow the filename convention “1.jpg”, “2.jpg”, etc.

Functions Allowed

For this project, imfilter, conv2, imrotate, im2double, rgb2gray, fspecial, imtransform, imwarp (and imref2d), meshgrid, sub2ind, ind2sub and all other plotting and matrix operation/manipulation functions are allowed.

Submission Guidelines

We will deduct points if your submission does not comply with the following guidelines.

Please submit the project once for your group – there’s no need for each member to submit it.

File tree and naming

Your submission on Canvas must be a zip file, following the naming convention YourDirectoryID_proj2.zip. For example, xyz123_proj2.zip. The file must have the following directory structure:

  • YourDirectoryID_proj2.zip/
    • Code/
      • MyPanorama.m
      • *(any dependencies of MyPanorama.m)
    • Images/
      • input/
      • CustomSet1,2/
      • (all of the other train and test images)
    • report.pdf

When run, MyPanorama.m must load a set of images from Images/input/, and return the resulting panorama.

Report

You will be graded primarily based on your report. We want you to demonstrate an understanding of the concepts involved in the project, and to show the output produced by your code.

Logistics and bookkeeping you must include at the top of your report (-5 points for each one that’s missing):

  • The name of each group member. - A brief (one paragraph or less) description of what each group member contributed to the project.

Include visualizations of the output of each stage in your pipeline (as shown in the system diagram on page 2), and a description of what you did for each step. Assume that we’re familiar with the project, so you don’t need to spend time repeating what’s already in the course notes. Instead, focus on any interesting problems you encountered and/or solutions you implemented.

Be sure to include the output panoramas for all five image sets (from the training and test sets). Because you have limited time in which to access the “test set” images, we won’t expect in-depth analysis of your results for them.

As usual, your report must be full English sentences, not commented code. There is a word limit of 1500 words and no minimum length requirement.

Collaboration Policy

We encourage you to work closely with your groupmates, including collaborating on writing code. With students outside your group, you may discuss methods and ideas but may not share code.

For the full collaboration policy, including guidelines on citations and limitations on using online resources, see the course website.

Acknowledgements

This fun project was inspired by a similar project in UPenn’s CIS581 (Computer Vision & Computational Photography).