Photo Manipulation, The Easy Way

Abstract

The typical smartphone user has many thousands of photos in their personal collection. Photo acquisition is effortless, and the next challenge is in devising methods to easily edit such large collections. Specifically, we need manipulation algorithms that are powerful enough for experts, yet simple for novices to master.

We identified three key directions to empower novice users with expert-level editing capabilities while maintaining an overall simplicity in the process. Those directions are (1) better selection masks, (2) high-level goal-centric algorithms and (3) domain specific algorithms. In this thesis we give examples from each category.

Given a photo, a novice user will typically either not edit it at all, or apply a simple global operation such as exposure correction. In contrast, a professional photo editor might perform local edits, specifying a selection mask to limit the operation to specific photo regions, or combining regions from several photos into a single composition. To ease selection mask creation we present a new patch embedding technique that allows for single-click selection masks.

Novices often think in terms of goals (e.g. improve lighting, de-clutter photo) and less in technical terms such as color spaces and image layers. One example of a high-level goal is the removal of distracting elements from photos. The task is motivated by the way professional photographers operate. They carefully frame the scene and might move objects around in order to stage the perfect photo. We define “photo distractors” as the elements that, if removed, would improve the photo. Using a simple slider interaction we allow users to automatically remove such distractors from photos.

It is at times useful to tailor solutions to specific photo types. As an example we show that, specifically for human heads, simple controls can induce sophisticated edits. Given a single portrait photo as input, we can change the pose of the head and the camera distance. This allows users to correct the “selfie effect”, i.e. big noses and small ears or to transform distant photos into selfies.

We conclude by discussing how each of these directions can be further explored to enable better image editing tools for novices.