A European Informational Website
learn more
A demosaicing algorithm is a digital image process used to interpolate a complete image from the partial raw data received from the color-filtered image sensor internal to many digital cameras in form of a matrix of colored pixels. Also known as CFA interpolation or color reconstruction, another common spelling is demosaicking.
Many modern digital cameras provide a raw file with data in a filter-mosaic format; the user can demosaic it using software, rather than using the camera's built-in firmware.
There are a number of different ways the pixel filters are arranged in practice, the most common being the Bayer filter as in the image at the right, alternating values of Red (R) and Green (G) for odd rows and alternating values of Green (G) and Blue (B) for even rows.
Since each pixel of the sensor is behind a color filter, the output is an array of pixel values, each indicating a raw intensity of one of three primary colors. Thus, an algorithm is needed to estimate for each pixel the color levels for all color components, rather than a single component.
Some methods may produce better results for natural scenes, and some for printed material, for instance. This reflects the inherent problem in estimating pixels that we do not really know for certain.
Naturally there is also the ubiquitous tradeoff of speed versus quality of estimation.
To reconstruct a color image from the data collected by the color filtering array, you need to fill in the blanks. The mathematics here is subject to individual implementation, and is called demosaicing. If you have a RAW image, you can use different demosaicing than what is built into the camera, often yielding higher quality.
In this example, we use Adobe Photoshop's bicubic interpolation to simulate the circuitry of a Bayer filter device such as a digital camera. In a typical commercial implementation, low pass anti-alias filters will be added that make the artifacts shown here less pronounced, with a corresponding reduction of sharpness.
This is the original image, made with Adobe Illustrator. On a perfect digital camera with perfect optical lens that can measure every pixel's exact Red, Green, and Blue components, the resulting image will probably look like this.
This is a simulated sampling taken by a Bayer filtered sensor array. Each pixel only has a value of either R or G or B.
A digital camera has certain circuits to reconstruct the whole image using above information. The resulting image could be something like this:
The reconstructed image is accurate in uniform-colored areas, but has a loss of resolution (detail and sharpness) and has edge artifacts (for example, the edges of letters have visible color fringes and some roughness).
These algorithms are examples of multivariate interpolation on a uniform grid, using relatively straightforward mathematical operations using only nearby instances of the same color component. The simplest is the bilinear interpolation method. In this method, the red value of a non-red pixel is computed as the average of the adjacent red pixels, and similar for blue and green.
These are algorithms that first compute an alternate representation, and use that to interpolate the color components.
These algorithms adapt their method of estimation (i.e. the estimation formula) depending on features of the area surrounding the pixel of interest.
Various commercial products implement proprietary estimation methods about which little is publicly known, and which may or may not be similar to publicly known algorithms.
Examples: