Sketch2Photo: Internet Image Montage

Tao Chen¹  Ming-Ming Cheng¹  Ping Tan²  Ariel Shamir³  Shi-Min Hu¹

¹TNList, Department of Computer Science and Technology, Tsinghua University
²National University of Singapore    ³The Interdisciplinary Center

Abstract

We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found by searching the Internet. Although online image search generates many inappropriate results, our system is able to automatically select suitable photographs to generate a high quality composition, using a filtering scheme to exclude undesirable images. We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images. Experimental results show the method is very successful; we also evaluate our system using the results from two user studies.

Paper

Sketch2Photo: Internet Image Montage
ACM SIGGRAPH ASIA 2009, ACM Transactions on Graphics, to appear
Tao Chen, Ming-Ming Cheng, Ping Tan, Ariel Shamir, Shi-Min Hu

System Pipeline

Retrieval Results

Composition Results

Video

Supplementary Materials

1. Intermediate results

General supplementary materials, including intermediate results and comparisons. [pdf]

2. User studies

High resolution compositions and detailed statistics of the user studies. [pdf]

Sktech2Photo Team

Tao Chen, Kun Xu, Fang-Lve Zhang, Meng Ding and Ming-Ming Cheng

Update: A web-based Sketch2Photo application: click here (Chinese), collaborated with Tencent.

Acknowledgments

This work was supported by the National Basic Research Project of China (Project Number 2006CB303106), the National High Technology Research and Development Program of China (Project Number 2009AA01Z330) and FDCT, Macau (Project Number 008/2008/A1). Tan Ping is supported by Singapore FRC Grant R-263-000-477-112.

Note

Original Name: PhotoSketch. We change the name since it's conflict with existing software: http://www.brainstormllc.com/.