This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Photosketcher: Interactive Sketch-Based Image Synthesis
Nov.-Dec. 2011 (vol. 31 no. 6)
pp. 56-66
Mathias Eitz, Technical University of Berlin
Ronald Richter, Technical University of Berlin
Kristian Hildebrand, Technical University of Berlin
Tamy Boubekeur, Telecom ParisTech
Marc Alexa, Technical University of Berlin
Photosketcher is an interactive system for progressively synthesizing novel images using only sparse user sketches as input. Photosketcher works on the image content exclusively; it doesn't require keywords or other metadata associated with the images. Users sketch the rough shape of a desired image part, and Photosketcher searches a large collection of images for it. The search is based on a bag-of-features approach that uses local descriptors for translation-invariant retrieval of image parts. Composition is based on user scribbles: from the scribbles, Photosketcher predicts the desired part using Gaussian mixture models and computes an optimal seam using graph cuts. To further reduce visible seams, users can blend the composite image in the gradient domain.

1. J. Hays and A.A. Efros, "Scene Completion Using Millions of Photographs," ACM Trans. Graphics, vol. 26, no. 3, 2007, article 4.
2. A. Agarwala et al., "Interactive Digital Photomontage," ACM Trans. Graphics, vol. 23, no. 3, 2004, pp. 294–302.
3. M. Eitz et al., "Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors," to be published in IEEE Trans. Visualization and Graphics, 2011.
4. J. Sivic and A. Zisserman, "Video Google: A Text Retrieval Approach to Object Matching in Videos," Proc. 9th Int'l Conf. Computer Vision, IEEE CS Press, 2003, pp. 1470–1477.
5. C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
6. Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, 2004, pp. 1124–1137.
7. P. Pérez, M. Gangnet, and A. Blake, "Poisson Image Editing," ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 313–318.
8. C. Rother, V. Kolmogorov, and A. Blake, "'Grab-Cut'—Interactive Foreground Extraction Using Iterated Graph Cuts," ACM Trans. Graphics, vol. 23, no. 3, 2004, pp. 309–314.
9. T. Chen et al., "Sketch2Photo: Internet Image Montage," ACM Trans. Graphics, vol. 28, no. 5, 2009, article 124.
10. J.-F. Lalonde et al., "Photo Clip Art," ACM Trans. Graphics, vol. 26, no. 3, 2007, article 3.
1. V. Kwatra et al., "Graphcut Textures: Image and Video Synthesis Using Graph Cuts," ACM Trans. Graphics, vol. 22, no. 3, 2003, pp. 277–286.
2. A. Agarwala et al., "Interactive Digital Photomontage," ACM Trans. Graphics, vol. 23, no. 3, 2004, pp. 294–302.
3. C. Rother et al., "AutoCollage," ACM Trans. Graphics, vol. 25, no. 3, 2006, pp. 847–852.
4. M. Johnson et al., "Semantic Photo Synthesis," Computer Graphics Forum, vol. 25, no. 3, 2006, pp. 407–413.
5. J.-F. Lalonde et al., "Photo Clip Art," ACM Trans. Graphics, vol. 26, no. 3, 2007, article 3.
6. T. Chen et al., "Sketch2Photo: Internet Image Montage," ACM Trans. Graphics, vol. 28, no. 5, 2009, article 124.
7. M.K. Johnson et al., "CG2Real: Improving the Realism of Computer Generated Images Using a Large Collection of Photographs," IEEE Trans. Visualization and Computer Graphics, vol. 17, no. 9, 2011, pp. 1273–1285.
8. M. Eitz et al., "Photosketch: A Sketch Based Image Query and Compositing System," Siggraph 2009: Talks, ACM Press, 2009, article 60; http://cybertron.cg.tu-berlin.de/eitz/pdf 2009_photosketch.pdf.
1. J. Sivic and A. Zisserman, "Video Google: A Text Retrieval Approach to Object Matching in Videos," Proc. 9th Int'l Conf. Computer Vision, IEEE CS Press, 2003, pp. 1470–1477.

Index Terms:
image databases, image retrieval, image segmentation, image synthesis, sketching, computer graphics, graphics and multimedia
Citation:
Mathias Eitz, Ronald Richter, Kristian Hildebrand, Tamy Boubekeur, Marc Alexa, "Photosketcher: Interactive Sketch-Based Image Synthesis," IEEE Computer Graphics and Applications, vol. 31, no. 6, pp. 56-66, Nov.-Dec. 2011, doi:10.1109/MCG.2011.67
Usage of this product signifies your acceptance of the Terms of Use.