Understanding & Using the RAW File Format

by | May 1, 2008

© Ron DayModern digital cameras require us to choose a file format for saving our images. The JPEG and RAW formats are the most common choices, but which one to use has sparked debate and led to confusion. This article examines the RAW file format, and provides guidance on how to use it to improve the quality of digital photographs.

I. The RAW Format

A digital RAW file is simply what its name implies, a file containing the unprocessed raw data captured by the sensor in the digital camera at the time of exposure. The RAW file does not contain the image in a form usable by most software. To actually acquire a usable image, the RAW file must first be converted. Camera settings for color space, sharpness, saturation, and white balance do not affect the image data in the RAW file. Rather, they are tags, which accompany the RAW file through the conversion process.

Each camera manufacturer created its own unique RAW format: Nikon .NEF, Canon .CRW and .CR2, Minolta .MRW, Olympus .ORF, Fuji .RAF, and the list goes on. As noted above, before exposed raw data in these formats is transformed into an image, it must first undergo conversion. Conversion can only be accomplished using the proprietary software supplied by the specific camera manufacturer, or purchased from a third party like Adobe, Breeze Systems, Bibble Labs, or Phase One. As an example, the interface of Nikon’s proprietary RAW conversion software, Capture NX, is shown in Figure 1.

Figure 1

Figure 1. Nikon Capture NX Interface

In sum, the RAW file contains mere data, which is processed by the photographer with special software before it is converted into an image. To draw an analogy, the RAW file is similar to a frame of exposed film waiting to be developed.

II. The RAW Advantage

When compared to the JPEG file format, the RAW format has some significant advantages.

1. NO LOST DATA. JPEG files are convenient because they are small. To become small, JPEG files are compressed. And, when they are compressed, a certain amount of file data is discarded. The JPEG format is known as a “lossy” format, because the more the file is compressed the more data it loses. A JPEG image may look fine when it is initially opened, but if it is opened and saved often, it starts to show degradation in quality.

The RAW file does not lose data. When converting a RAW file into an image, most photographers save the image in TIFF or PSD (Photoshop) format. These formats are referred to as “loss-less,” because they preserve image quality and lose no data irrespective of the number of times they are opened and saved.

2. CONTROL OF DATA. In the JPEG format, an in-camera computer is programmed to convert the raw data captured by the sensor into an image based on the settings (exposure, contrast, sharpening, saturation, white balance, etc.) existing when the picture is made. During this process the in-camera computer, not the photographer, applies a tone curve to the data in an attempt to create an image with acceptable brightness and contrast levels. The original raw data captured by the sensor is altered, and is no longer available.

RAW file conversion, on the other hand, allows the photographer to process all of the original data on a desktop or laptop computer, which has considerably more speed and power than the in-camera version. The photographer is able to change the camera settings for color space, contrast, sharpening, saturation, white balance, and, to a limited degree, exposure, just as though making or adjusting these settings before taking the picture. And, since the raw data is converted on the photographer’s computer, the effect of these adjustments can be observed in real time on screen.

Also, the photographer is always free to return to the RAW file to change settings and process the image differently whenever necessary, because RAW conversion does not alter the underlying data. The original RAW file is preserved.

Figure 2

Figure 2. White Balance – Adobe Camera Raw 4.3, Photoshop CS3

3. WHITE BALANCE. The color temperature of the light, and often the mood of an image, is controlled by white balance. The control over this setting in the RAW format is significant. If a photographer incorrectly sets the white balance on a camera prior to taking a picture in JPEG format, the die is cast. The only hope lies in an attempt to adjust color, hue, and/or saturation in Photoshop to correct the error, and there is no guarantee of success.

However, during the process of converting a RAW file into an image, the photographer may reset the white balance to any specific value, just like resetting the white balance before the exposure, without any loss or damage to the underlying data. In fact, in Adobe Camera Raw, one can fine-tune the color temperature of the light to the degree. See Figure 2. Viewing the differences between various white balance settings on screen is like being able to try out a white balance setting before taking the picture. That’s impressive.

4. BIT DEPTH. The quality of a digital image is often related to its bit depth, because as bit depth increases the number of possible tonal values that can be recorded grows exponentially. JPEGs are 8-bit files, and most RAW files are 12-bit. An 8-bit file can measure 256 tonal values in each of the three-color channels, or a total of 16.7 million possible colors per pixel. A 12-bit RAW file, however, can measure 4,096 tonal values per channel, or a total of 68.7 billion possible colors per pixel.

A JPEG file, therefore, records considerably fewer tonal values than a RAW file. The tonal values that it does not record are lost forever. As a result, the tones in the JPEG file format are sometimes not smooth, a condition referred to as “posterization.” An example of posterization can be seen in Figure 3. It often occurs in areas of the sky in an image where the change in tones is very gradual and subtle. Tonal gradations are much smoother, and image detail is more accurate in 12-bit RAW files that discern many more color tones per pixel than an 8-bit JPEG.

Figure 3

Figure 3. Example of posterization

By the way, when converting a RAW file, it is recommended to save it as a 16-bit file (12-bit is not available) in either TIFF or PSD format. This way, all of the data in the image following conversion is preserved in a stable file format that will not lose data when opened, closed, or compressed. Photoshop CS3 offers considerable support for processing these 16-bit files.

5. EXPOSING RIGHT. Many photographers are paranoid about blowing the highlights in a scene, and for good reason. Once the highlights are truly clipped, they cannot be recovered.

To avoid this dilemma, it appears logical to underexpose a scene, and later recapture the brightness in processing.

However, a tradeoff soon becomes evident. As one brightens parts of a scene in shadow, one often introduces “noise” into the image. The sensor signals causing noise are normally of low intensity and are recorded in the shadows, which are displayed on the left side of the histogram. Some compare noise in digital images to the grain seen in film. Both increase as light levels decrease.

A further issue is how the tonal values in the raw image data are allocated from the highlights down to the shadows. If the image data is grouped into zones equivalent to exposure stops, we find that the first stop, containing the brightest tones, records 50% of all the tonal values captured on the sensor. Figure 4 shows the breakdown of recorded tonal values assuming a digital camera with a dynamic range of about five exposure stops.

Figure 4

Figure 4.

This means in a 12-bit RAW file with 4,096 tonal values per channel, that one-half of those values, or 2,048, are recorded in the brightest pixels on the right one-fifth of the histogram. The fourth exposure stop, which contains dark tones, records a mere 256 tonal values. The fifth stop on the left one-fifth of the histogram, containing the darkest tones, records only 128 values. The absence of an adequate number of tonal values being recorded in the shadows explains why noise is often generated when shadows are significantly brightened. It can also explain how posterization can develop in dark regions of the image that should show smooth tonal gradations.

This can be an important issue. In his tutorial, Michael Reichmann points out, “that if you do not use the right fifth of the histogram for recording some of your image, you are in fact wasting fully half of the available encoding levels of your camera.” Another well known photographer, John Shaw, also recognizes this principle, and warns: “[u]nderexpose your image by only one stop when shooting RAW and you’ve lost half of all the possible information.” John Shaw’s Photoshop Guide, ch. 2, pg. 13.

The solution is to use as much of the sensor’s recording range as possible. Expose images so the right side of the graph in the histogram comes as close as possible to the right margin without clipping the highlights. This maximizes image quality, insuring smooth tonal transitions and reducing noise to the extent possible within a single exposure. Although the image may appear too bright when first opened in the RAW conversion software, adjustments are easily made to lower the brightness or increase contrast without degrading image quality, so the final image satisfies your discriminating eye.

While the “expose right” technique can be applied somewhat with JPEG images, it is best used with RAW files to record the maximum number of tonal values and maintain higher quality during RAW conversion and later processing.

III. A RAW Comparison

The RAW file format sounds good, but does it actually produce? Can one see a meaningful difference between a processed RAW file and a processed JPEG file of the same image? To find out I conducted a test.

I was particularly interested in differences between the formats when processing underexposed and overexposed images. So, in a series of images, I underexposed my test subject by 1 EV, 2 EV, and 3 EV, and then overexposed it by 1 EV, using both RAW and JPEG file formats.

In processing, I added or subtracted the requisite exposure (EV) necessary to bring each image back to same exposure. The RAW files were processed in Adobe Camera RAW (ACR), and the JPEG images were processed in Photoshop. Then sections of the images were compared at 100%. The results are quite interesting.

IV. The RAW Converter

As noted previously, before a RAW file can be transformed into an image, it must first undergo conversion with the proprietary software supplied by the specific camera manufacturer, or purchased from a third party like Adobe, Breeze Systems, Bibble Labs, or Phase One.

In Photoshop 7, Adobe first introduced, as a “plug-in,” an optional RAW image converter. It possessed the proprietary information for processing the RAW files made by most cameras. Today, this converter, Adobe Camera RAW, comes with Photoshop CS3 and Photoshop Elements 5. ACR has greatly streamlined the workflow of many digital photographers who have adopted it as their RAW file converter of choice. Its interface is shown in Figure 5.

Figure 5

Figure 5. Adobe Camera RAW 4.3.1 Interface – Photoshop CS3

The magnifying glass at the top left allows the image to be enlarged so image quality may be analyzed. In the same area, crop, rotation and other tools are available. Color space, bit depth and file size are selected and controlled in the bottom center area. At the top right note the histogram, followed by a menu for selecting White Balance, and sliders for adjusting Color Temperature and Tint.

The Exposure slider controls the highlights on the right side of the histogram, while the Blacks slider controls the dark values to the left. The Brightness slider is used to adjust mid-tones. The Recovery and Fill Light sliders control highlight and shadow recovery. Sliders for adjusting Contrast and Saturation also are provided.

Clicking on the various icons located above White Balance provides the photographer with controls for Tone Curves, Sharpness and Noise Reduction, HSL/Grayscale, Split Toning, Lens Correction, Camera Calibration, and Presets. Also, note immediately below the histogram that the ISO, shutter speed, aperture, and lens focal length are conveniently listed.

When you complete processing the image in ACR, and it looks the way you want, simply click on “Open,” at the lower right, and the converted image will open in Photoshop ready for further processing, or saving.

V. Conclusion

Whether to shoot in the JPEG or the RAW format depends on your needs as a photographer. If you want images for snapshots, to e-mail to friends, to share over the internet, or even to wire to an employing newspaper to meet an overnight deadline, then the smaller compressed file size offered by the JPEG is excellent.

Also, if an image is captured at the largest file size, with the detail setting at “maximum,” a good color print can be made from a JPEG file. This assumes the color balance and exposure are correct.

On the other hand, if you enjoy processing your own images, or if you don’t always get the exposure and white balance right, then the flexibility of the RAW format is appealing. RAW files are larger than JPEGs, so they fill more space on a flash card and on a hard drive, but their advantages are significant.

If you are after valuable, hard-to-get images, the image of a lifetime, or if you want to reproduce your images in quality publications or as quality prints, then the RAW format is almost mandatory. It records so many more colors and tonal variations than a JPEG. Prints reveal more detail, have smoother tonal gradations and avoid posterization, which is sometimes found in JPEGs.

Post a Comment

Logged in as Anonymous