PhotoPNMTools

More information about photography and imaging| About gamma correction| About the Y value About sharpness and sharpening

I am slowly developing a toolset for processing photographic images.

PhotoPnMTools now includes interactive editing of pictures and generation of Web albums.

SEE THE README FILE

Download Version 1.3 (Changes)
Download Version 1.2.1
Download Version 1.2 (patchlevel 2,fixed rounding error problem)
Download Version 1.1
Download Version 1.0

To whet your appetite: some test images!

Since I got into digital photography, I found out much about image processing that I found quite shocking. It is my experience that some of the most basic algorithms found in popular image processing software are implemented quite badly. It is, however, not very hard to come up with improved algorithms.

pnmresample: shrinking and interpolating

Shrinking

Various software does a bad job of shrinking images. Consider shrinking an image by a factor of 2 or more. What they do is just re-sample: take samples out of the original image, for example, they take every other pixel and ignore the values of the remaining pixels. Some use interpolation algorithms (such as bilinear, bicubic, etc), but that only makes any difference for non-integer scaling factors. What is entirely missing here is an algorithm that takes into account all source pixels, instead of throwing away three-quarters or more of them. This is called anti-aliasing. A first naive solution comes to mind: make each destination pixel an average of the source pixels that it encompasses. With a scaling factor of 2, each 2x2 area of pixels is averaged to form one pixel in the destination image. This is what for example xv does. This, however, is still not ideal, as the precise position of each within the destination pixel is not accounted for. This is especially visible when scaling with a non-integer factor, but it also results in very strong artifacts in some line drawings (one of the worst cases is an alternating pattern of black and white pixels: this produces either grey, or a similar black-white pattern, depending on the alignment of the pixels).

A better solution is to weigh the source pixels in the destination image according to their relative position to the four closest destination pixels. I first tried this algorithm to render fractals, with very good results. While the perceived sharpness may be less than without anti-aliasing (i.e. details have a somewhat soft contrast), this is misleading, as the method retains significantly more visually-useful detail than without anti-aliasing. The weights can be made non-linear, or one can use a sharpening filter, to compensate for the soft detail contrast.

Theoretically, an anti-aliased image does not contain more information than a non-anti-aliased one, but in practice, an anti-aliased image is visually more meaningful: the resulting image looks as if you were looking at it from such a distance that the details look blurred. There are some cases in which anti-aliasing shows very much improved results. For example, the effect of anti-aliasing really shows in fractals. I did a really wonderful fractal print by shrinking a large image, and sharpening it afterwards.

See test images here

Enlarging

Enlarging an image is another matter, and requires different techniques. Various software provides interpolation algorithms, which are suitable for this purpose. There is, however, also a `theoretically ideal' interpolation method that is not found very often: the two-dimensional sinc interpolation. So, I decided to implement this method in pnmresample. When pnmresample is used to expand an image, it switches to the sinc algorithm. It can be parametrised, with the most degenerate settings producing an effect similar to bilinear interpolation.

Sinc also has a disadvantage: it produces ripples, sometimes referred to as `ripple artifacts', even though they are not, strictly speaking, artifacts, but rather, the result of a wrong assumption. The assumption is that the original image of which the sampled image was taken does not contain signals of greater frequencies than could be recorded by the sampled image. While this assumption is not necessarily false, it usually is in practice. I did not find the ripples very obtrusive, though, and they do produce some extra sharpness to the image, which is what they are for. They are worst in line drawing type images, and least obtrusive in photographic images. When applying sharpening filters though, they may become obtrusive in some cases. For large magnifications, an advantage is that the pixels of the original image are minimally visible.

See test images here

Rotating

pnmresample should eventually support arbitrary angle rotation using sinc.

ppmhsy: Editing brightness and colour characteristics

Ppmhsy uses a variant of the Hue, Saturation, Brightness type colour spaces. They are well known, and are especially useful for changing saturation, which is something I find myself doing often. What the software does is translate the R,G,B values into hue,saturation,brightness triplets, then modify some values in the triplets, then translate them back. As it turns out, different software uses different models: HSV, HSI, HLS, LHS, CIEHLS, etc. Particularly common are HSV and HLS. Though both are far from ideal, especially HLS is often used for saturation tools. Some tools (such as xv) use HSV.

Brightness

Most of these models have a significant flaw: the brightness (called Lightness, Luminance, Brightness, Value, Intensity, etc) parameter is not the same as the perceived brightness. HSV uses Value = MAX(R,G,B), HLS uses lightness = MAX(R,G,B) + MIN(R,G,B). Better is HSI, which uses Intensity = R+G+B. But, still better would be a weighted sum of R, G, and B, as red, green, and blue have different relative brightness. I experimentally obtained the following Y:

Y = R*0.211 + G*0.658 + B*0.131

Y is a dimension found in the colour space used by TVs, and may also be used to translate a colour image into a greyscale image, and produces about the right intensities for the different colours (for example, blue is darkest and yellow is lightest). Now, pnmhsy uses this weighted sum as its brightness parameter. The weights can be user-defined, but the defaults are those given above.

Saturation and whitening

In HSY, saturation is defined in terms similar to that of HSV: pure colours have high saturation, colours with grey or white mixed in have lower saturation. This is unlike HLS, which uses two different formulas for saturation, one for low brightness levels and one for high brightness levels. For low brightness levels, the formula is similar in spirit to HSV. For high brightness levels, it uses a formula that ensures that an image that is close to white has high saturation (i.e. bright pink (r=255,g=200,b=200) has saturation=1, while in HSV, this is just pink, and has the same saturation as a darker pink. Note that when an image is darkened, the HLS saturation diminishes). I did not find any mention about why HLS uses this seemingly illogical, non-linear formula for saturation. But it appears that this is a way to ensure that bright colours close to the maximum white (r=255,g=255,b=255) remain bright, and do not darken when saturation is increased. For example, our pink (255,200,200) can become more saturated only when decreasing the green and blue (for example (255,100,100)), but this implies darkening the colour. This `darkening' when an image saturates too far is a well-known property of the HSV model, and is thus avoided in the HLS model.

However, when we look at it more theoretically, there's a neater solution to the `darkening' problem. If we look again at our bright pink (r=255,g=200,b=200) we would expect an increase in saturation to lead to something like (r=350,g=150,b=150) or so, while in HSV we get (r=255,g=150,b=150), and in HLS the colour remains the same, as saturation cannot further be increased. In our HSY model, we do get something like (r=350,g=150,b=150). To ensure that we get legal rgb values again (r,g,b <= 255), HSY can use two ways: decrease brightness to make the rgb fit in (darkening), or add some white back to the colour to simulate a very bright colour (called whitening). We can darken manually by applying some darkening (-d), and whitening is done automatically (but can be turned off or decreased using -w).

If we look again at the HLS formula again from the perspective of whitening as a means to compress rgb values back to the legal range, we could say that whitening does about the same thing as HLS for bright colour, only it is theoretically sounder and (I think) it looks better. Actually, we could say that HLS assumes that bright colours (such as our bright pink) are actually colours which were originally very bright and saturated, and are represented in the image as whitened. But, we will never know if our bright pastel colours (such as our pink) are just that, or are actually overexposed areas which were whitened by the medium (actually film and digital cameras behave differently here, though both use a kind of whitening). Here, we assume for simplicity and mathematical neatness that colours are just what they are. However, we do have two different choices which are appropriate for these two different situations. If colours are just what they are, we can darken the image a little when we increase saturation, to obtain an accurate high-saturation image. If some colours represent over-exposed areas (such as the sun in a sunset) we don't use darkening, as it will make the over-exposed areas look greyish (a grey sun looks pretty unnatural).

One surprising thing I discovered was that the HLS model actually introduces serious noise into pictures. I first thought that this noise was just noise in the picture that is amplified, but as it turns out, it is noise resulting from the brightness problems with L. For some colours (such as cyan), small changes in H and S lead to great changes in brightness, even while L stays the same. When we saturate, noise enters in the picture. See the test page below.

Beside saturation and brightness, ppmhsy also turned out to be quite useful for changing colour balance and contrast in a natural way. I was in fact quite surprised by the wonderful results obtained by doing brightness-preserving colour balance using HSY, and doing contrast operations in HSY space. See the test page for results.

Illustration of some functions of ppmhsy
Saturation test images
Contrast test images

pnmconvolve: sharpening

I've started on my own sharpening/blurring filter. I use a standard 3x3 convolution sharpening, sharpening the finest details. This can be used to sharpen up an image which is sharp but in which the contrast in the finest details is a bit soft, such as an image that was softened after shrinking. Larger radiuses are particularly useful only if your image is not sharp.

The standard convolution filter has a couple of weaknesses, which I tried to remedy using a couple of special parameters. One weakness is that noise is amplified, and we especially see `white noise' in plain areas. This problem can be solved by having pixels with low contrast weigh less (similar to the `unsharp mask' threshold parameter, only continuous). A second weakness is that borders with high contrast are too much intensified. This problem is solved by reducing large changes in intensity of pixels. The result is a filter that still looks good at high sharpening factors.

pnmconvolve also supports unsharp mask sharpening. To do this, one specifies a blur operation, and then uses the blurred result as the unsharp mask.

Boris van Schooten, boris@13thmonkey.org