跳至主要內容

Content-aware image resizing in JavaScript

linwu大约 16 分钟

Content-aware image resizing in JavaScript

Content-aware image resizing in JavaScript
Content-aware image resizing in JavaScript

There is an interactive version of this postopen in new window available where you can upload and resize your custom images.

TL;DR

There are many great articles written about the Seam Carving algorithm already, but I couldn't resist the temptation to explore this elegant, powerful, and yet simple algorithm on my own, and to write about my personal experience with it. Another point that drew my attention (as a creator of javascript-algorithmsopen in new window repo) was the fact that Dynamic Programming (DP) approach might be smoothly applied to solve it. And, if you're like me and still on your "learning algorithms" journey, this algorithmic solution may enrich your personal DP arsenal.

So, with this article I want to do three things:

  1. Provide you with an interactive content-aware resizer so that you could play around with resizing your own images
  2. Explain the idea behind the Seam Carving algorithm
  3. Explain the dynamic programming approach to implement the algorithm (we'll be using TypeScript for it)

Content-aware image resizing

Content-aware image resizing might be applied when it comes to changing the image proportions (i.e. reducing the width while keeping the height) and when losing some parts of the image is not desirable. Doing the straightforward image scaling in this case would distort the objects in it. To preserve the proportions of the objects while changing the image proportions we may use the Seam Carving algorithmopen in new window that was introduced by Shai Avidan and Ariel Shamir.

The example below shows how the original image width was reduced by 50% using content-aware resizing (left image) and straightforward scaling (right image). In this particular case, the left image looks more natural since the proportions of the balloons were preserved.

Content-aware image resizing
Content-aware image resizing

The Seam Carving algorithm's idea is to find the seam (continuous sequence of pixels) with the lowest contribution to the image content and then carve (remove) it. This process repeats over and over again until we get the required image width or height. In the example below you may see that the hot air balloon pixels contribute more to the content of the image than the sky pixels. Thus, the sky pixels are being removed first.

JS IMAGE CARVER DEMO
JS IMAGE CARVER DEMO

Finding the seam with the lowest energy is a computationally expensive task (especially for large images). To make the seam search faster the dynamic programming approach might be applied (we will go through the implementation details below).

Objects removal

The importance of each pixel (so-called pixel's energy) is being calculated based on its color (R, G, B, A) difference between two neighbor pixels. Now, if we set the pixel energy to some really low level artificially (i.e. by drawing a mask on top of them), the Seam Carving algorithm would perform an object removal for us for free.

JS IMAGE CARVER OBJECT REMOVAL DEMO
JS IMAGE CARVER OBJECT REMOVAL DEMO

JS IMAGE CARVER demo

I've created the JS IMAGE CARVER