Adjust color levels with a color matrix in Android


Using a color matrix to correct levels in images

Recently we have been developing the WytePad Scan App. You can use it to take pictures of WytePads and it will automatically crop and transform the picture so that looks right and is easy to use in other applications such as the Marvel App. While mobile cameras are rapidly getting better they are still not great at taking pictures of black and white surfaces and preserving the right white balance and exposure. Our WytePads are mostly white with doodles on, so we wanted to find a way to automatically correct potentially bad images.

One simple way to correct exposure issues in images is to adjust the image levels. This is quickly done in Photoshop by using a “Level adjustment layer”. There you can see the histogram of the image and transform it by moving its lower and upper bounds. When you click confirm, the histogram is transformed so the selected upper and lower bounds become the start and end of the histogram. Read more about levels on Cambridge in colour.

Adjusting levels in Photoshop

BoofCV

We are using BoofCV for image recognition in the WytePad app, so we started by using the BoofCV class EnhanceImageOps for correcting the levels. However the process of transforming a histogram is not very well documented and it took multiple seconds to correct the color in a bitmap, which is not a great user experience.

RenderScript

In an attempt to optimize the speed we tried using RenderScript. To our luck there are some examples that demonstrates how to change levels in RenderScript like this blog post. This works quite well, however at least one allocation with the same size as the image is needed during processing which is taxing on the memory usage. As we do not want to split images into parts and would like as large an image as possible to be handled, we needed another solution.

Color Matrix

The color matrix documentation describes how a color matrix can be used to transform the value of each pixel in an image. It can then be converted into a color filter and applied to a Paint object or an ImageView. The process of applying a color matrix is hardware accelerated and highly optimized. Our experience was that this was way faster than using RenderScript.

As an example we described here how we are using the color matrix to change levels on a picture in the WytePad Scan App:

public class ColorCorrectionHelper {
    public static final float COLOR_MAX = 255;

    public static ColorMatrixColorFilter getCorrectLevelsFilter(Interval interval){

        float histogramWidth = interval.max - interval.min;
        float scale = COLOR_MAX / histogramWidth;
        float offset = interval.min * scale * -1;

        ColorMatrix cm = new ColorMatrix(new float[]
                {
                        scale, 0, 0, 0, offset,
                        0, scale, 0, 0, offset,
                        0, 0, scale, 0, offset,
                        0, 0, 0, 1, 0
                });

        return new ColorMatrixColorFilter(cm);
    }
}

public class Interval {
    public int min, max;
}

When used, the ColorCorrectionHelper takes an interval as a parameter. This interval describes the index of the new lower and upper bounds of the histogram. In a 8 bit bitmap as the one we use, the histogram has the length = 255. Each row in the matrix describes how one color should be transformed. To adjust the levels we create a color matrix that scales and offsets the value of each color.

The static method below describes how we find the histogram and retrieve the lower and upper bounds that should be used for the level correction. If you are processing large bitmaps it is advisable to first resize the bitmap before passing it to this method. ConvertBitmap.bitmapToGray and ImageStatistics.histogram are part of the BoofCV library we use.

   
static Interval calculateIntervalForColorCorrection(Bitmap bitmap) {
    ImageUInt8 gray = new ImageUInt8(bitmap.getWidth(), bitmap.getHeight());
    ConvertBitmap.bitmapToGray(bitmap, gray, null);

    int histogram[] = new int[256];
    ImageStatistics.histogram(gray, histogram);
    Interval interval = new Interval();

    //find lower bounds. We look for darkest entry in histogram
    for (int i = 0; i < histogram.length; i++) {
        if (histogram[i] > 0) {
            interval.min = i;
            break;
        }
    }

    //find the index of the level that is most most widely used in image.
    //This works as WytePads are all white
    int maxValue = 0;
    for (int i = histogram.length - 1; i >= 0; i--) {
        int value = histogram[i];
        if (value <= maxValue) {
            maxValue = value;
            interval.max = i;
        }
    }
    return interval;
}

The following short snippet shows how we apply the ColorFilter to a Paint object that is then used to draw a Bitmap onto a canvas.

ColorMatrixColorFilter correctLevelsFilter = ColorCorrectionHelper.getCorrectLevelsFilter(mHistogramInterval);
mColorCorrectionPaint = new Paint();
mColorCorrectionPaint.setColorFilter(correctLevelsFilter);

...

canvas.drawBitmap(bitmap, bitmapRect, wytePadInnerRect, mColorCorrectionPaint );

The resulting color level correction can be seen in the image below - original to the left and the greatly improved result to the right.

Color level correction comparison



By: Tobias Alrøe