Page 1 of 2

Histogram myth, your thoughts!

Posted: Sun Feb 19, 2012 5:28 pm
by Greg Basco
Hi, everyone. I've been having a very interesting exchange with a gentleman who will be a client of mine on a photographic workshop here in Costa Rica in a couple of months. This person has a seriously solid background in imaging science, and he's been educating me on his position about how the oft-repeated reason for exposing to the right (that the rightmost fifth of the histogram has half of the potential tonal values and the allegedly resulting loss of 1/2 more tonal values for every stop of underexposure) is not logical because the histogram is by its very nature, linear. I've come around to his way of thinking and wanted to see what some of the tech folks here thought about it.

Here's this gentleman's article: http://photomorrobay.wordpress.com/

For those of you who may not know the background on the claim, these links will explain what I and many others have taken to be common knowledge in digital photography today:

http://www.luminous-landscape.com/tutor ... ight.shtml
http://www.naturescapes.net/docs/index. ... -histogram
http://www.rondayphotography.com/Unders ... Format.htm
http://www.deepgreenphotography.com/201 ... age-files/

I'm really interested in the responses as the debate about the validity of this concept is quickly starting to go over my non-tech head :)

Cheers,
Greg

PS -- If the mods think this topic would be better off in a different forum, please move as you see fit.

Posted: Sun Feb 19, 2012 6:17 pm
by Joseph Martines
I am interested to see how others respond to this.

I'm not a techie so my comprehension is limited.

Posted: Sun Feb 19, 2012 6:44 pm
by Greg Downing
The way I understand it, with my limited techie head, is that while a histogram may be a linear representation of the data the actual data itself is not.

Posted: Sun Feb 19, 2012 6:49 pm
by Greg Basco
Greg, you mean the data when converted to RAW?

Cheers,
Greg

Re: Histogram myth, your thoughts!

Posted: Sun Feb 19, 2012 8:18 pm
by Ozark Bill
This is an interesting topic, and I will look forward to finish reading this link. In my perspective, avoidance of noise is the primary benefit from shooting to the right. In my experience shooting to the right has enabled me to get much better results at high ISO's than if I had gone for the "perfect exposure". The difference is between something great and something unusable. (I use Canon 50D, 7D and P&S cameras)

Posted: Sun Feb 19, 2012 10:47 pm
by StephenFitzpatrick
Why theorize? Give it a try.

Posted: Mon Feb 20, 2012 1:03 am
by E.J. Peiker
You might give this a read from one of the leading experts on the planet in RAW conversion and digital photography technology:
http://schewephoto.com/ETTR/index.html

Posted: Mon Feb 20, 2012 1:28 am
by ahazeghi
The image sensor is always linear with light intensity, however a curve is always applied to RAW data to make it look natural since human precipitation is logarithmic and not linear. the histogram is derived from the curved and demosaiced data so it depends on the cameras processing parameters (it is a processed JPEG histogram) and may not accurately represent the RAW data recorded by the sensor.

Posted: Mon Feb 20, 2012 8:06 am
by Greg Basco
Hi, folks. Thanks for the responses so far. Please note that the argument is not that exposing to the right isn't good practice. The gentleman making this argument is not against the practice and realizes the practical benefits of doing so. The point is that the claim that the "brightest fifth of the histogram holds 50% of all tonal information..." may not be true.

Arash, thanks for your reply. I was hoping you'd see this. Can you elaborate on the curve applied to RAW data and how that might affect the argument? I'm interested to read more.

Cheers,
Greg

Posted: Mon Feb 20, 2012 8:28 am
by E.J. Peiker
I do agree with the fifth comment because that assumes a dynamic range of just 5 stops. This was true bak in the days of the EOS 1D or Nikon D1 but isn't today.

Re:

Posted: Mon Feb 20, 2012 9:39 am
by Gray Fox
Greg Basco wrote:The gentleman making this argument is not against the practice and realizes the practical benefits of doing so. The point is that the claim that the "brightest fifth of the histogram holds 50% of all tonal information..." may not be true.
Greg, I think your friend might have avoided confusion on the part of readers if he had more clearly emphasized up front the point that ETTR as an exposure strategy is separate from the linearity of the histogram display. Greg D. was the first to point out that ambiguity. As it is, the article can certainly be misconstrued by the less technically savvy. And, I have to confess, the “histogram myth” never occurred to me. Adjusting exposure results in a histogram change that is, if not precisely linear, close enough so as not to be attention getting. Clearly it isn’t logarithmic. One would notice such things! :wink:

Posted: Mon Feb 20, 2012 9:47 am
by Greg Basco
Michael, good point about the article's wording. The gentleman's beef is indeed with the underpinning, not with ETTR as good practice.

Cheers,
Greg

Posted: Mon Feb 20, 2012 10:59 am
by dbostedo
I've not had to really read the Morro article. According to several things I've read, the original Luminous Landscape article is essentially wrong. The main reason to use ETTR is simply to maximize the overall signal to noise ratio of the image. More light = higher signal = higher SNR = cleaner picture. The concept of it allowing more levels is not what makes ETTR yield improved images.

And if anyone would like a more practical take on it (from someone who just thinks you shouldn't worry so much about noise anymore), you can read this article from Ctein at The Online Photographer.

Posted: Mon Feb 20, 2012 11:19 am
by Greg Basco
Hi, David. That's very interesting. Do you have any links to those articles that countered Michael Reichmann's original article? I'm curious to learn more about this whole thing. Thanks also for the link to the article at The Online Photographer -- interesting as well.

Cheers,
Greg

Re:

Posted: Mon Feb 20, 2012 12:16 pm
by ahazeghi
Greg Basco wrote:Hi, folks. Thanks for the responses so far. Please note that the argument is not that exposing to the right isn't good practice. The gentleman making this argument is not against the practice and realizes the practical benefits of doing so. The point is that the claim that the "brightest fifth of the histogram holds 50% of all tonal information..." may not be true.

Arash, thanks for your reply. I was hoping you'd see this. Can you elaborate on the curve applied to RAW data and how that might affect the argument? I'm interested to read more.

Cheers,
Greg
Hi Greg,
If you have Canon DPP you can see what the linear sensor output looks like, open a CR2 file and then click linear on the RAW tab. This is what the sensor has recorded. now un-check linear and observe how the tone curve changes.you can also click on the RGB tab to see the RGB histogram.

ETTR increases SNR but it may degrade your highlight tones if you push it too much. I wouldn't say "brightest fifth of the histogram holds 50% of all tonal information..." that's not true because as I mentioned histogram is a result of the tone curve applied to RAW data and it is variable. The real RAW histogram is what is shown in the RAW tab in DPP and it is quite different from the RGB histogram you see in PS or LR.

ETTR is best for cameras with low DR, excessive pattern noise and small pixels like the 7D. With the 5D2 and the 1D I don't really push the exposure, just keep the highlights at check and I can deal with the shadows in post processing with excellent results. An yes, the "number of levels" argument is just nonsense, the ADC in the camera does not care about exposure. The RAW is always 14Bits.

Hope this helps

Posted: Mon Feb 20, 2012 12:47 pm
by dbostedo
A couple more sources Greg :

One section from Emil Martinec's excellent paper on noise (I'd suggest everyone read the whole thing - although parts of it are getting out of date and it IS pretty technical.)
Chromasoft 1
Chromasoft 2
Chromasoft 3
Chromasoft 4
Open Photography Forums thread (Not specifically about why ETTR is good)

Posted: Mon Feb 20, 2012 2:34 pm
by Greg Downing
Interesting read - and I was talking about the RAW data when processed. While I am not one to generally get tied up in the exact science or specific math on any of this stuff, I do know I get better noise performance when ETTR, but noticeably lesser so as digital imaging technology evolves.

Posted: Fri Feb 24, 2012 4:04 pm
by signgrap
This has been a very informative discussion. Certain aspects that were discussed start to go over my head being someone whose last physics class was more than 55 years ago in high school. Certain observations and assumptions that several contributors (both here and in the linked articles) require a technical frame of reference that is much greater than mine and I daresay many of this forums participants. So I will endeavor to put down the conclusions that I have come to as they relate to my "exposure" philosophy. I would like to use lay terms and not scientific jargon so that the least technical among us can understand the important information shared here. The more knowledgeable among you, please make any corrections for inaccuracies, omissions or to improve clarity. I am making a presentation to our local camera club and I want to make certain that the information I impart is accurate and easily understood.

As a generalization the Best Exposure advice is to make an exposure that is accurate for the given situation providing the exposure still retains detail, when needed, in the highlights. Typically super bright areas such as specular highlights and reflections from shiny surfaces can be ignored when assessing a histogram/exposure. This assumes that there are no important details in these areas and they comprise a relatively small area of the total image.

ETTR - Expose To The Right main benefit is reducing noise in the dark and shadow areas.

ETTR is beneficial just so long as there is sufficient highlight headroom to record the brightest areas of the image and still retain all detail in these bright areas. Since histograms don't necessarily show the brightest areas accurately (because the histogram is generated from the embedded JPEG not the RAW file) you need to be conservative in how far to the right you push the exposure. Blown detail in white/bright areas is to be avoided.

If ETTR means long shutter speeds in order to move the exposure to the Right you may actually be better off using the "normal" exposure because long shutter speeds add noise. (Noise thresholds vary between camera makes and models however; is there an "average" threshold "shutter speed longer than" where noise becomes a problem? E.g. 2 seconds and longer)

Michael Reichmann's article from the Luminous Landscape on ETTR from 2003 is now technically out of date. Sensors from 2002-3 had more noise and less Dynamic Range (DR) i.e. approximately 5 stops, compared with today's sensors. The sensors in today's D SLRs have an approximately 10 - 11 stop DR so the information imparted in that article is technically inaccurate in reference to current day high-end cameras.

When shooting RAW it is important to select the appropriate "Picture Style" usually something like "Standard" and then make a custom adjustment to this selection as follows:
Contrast adjusted to -3
Saturation adjusted to -3
Sharpness adjusted to +3
The Contrast and Saturation adjustments will help the embedded JPEG file more accurately reflect the greater DR of the RAW file. The end result of these two adjustments is making the histogram a better reflection of the DR in the RAW file. The Sharpness adjustment allows you to more easily determine focus accuracy by displaying a more highly sharpened image on the camera LCD. None of these adjustments affect the RAW file.

The one area that I am still unclear about is; how much information each section of the histogram has compared to its nearest neighbor (assuming the histogram is divided vertically into five equal areas). What I have gathered from this discussion is that Michael Reichmann's 2003 assertion that there is a doubling of information between adjacent areas of the histogram when reading from left to right is incorrect. In other words the first four sections of the histogram when combined do not contain less information than final fifth area to the right i.e. the brightest area. There seems to be some increase in information as we go from dark to light since human vision is better adapted to seeing brightness than it is discerning information in dark areas/at night. But what is unclear is whether there is any difference in the amount of data recorded in the dark vs. bright areas by the sensor i.e. how much information is recorded in the dark areas vs. the bright areas as it is represented on the histogram?

It was said that the sensor records information in a linear fashion. Does this mean that each section of the histogram is capable of recording the same amount of data?

Re:

Posted: Fri Feb 24, 2012 4:22 pm
by ahazeghi
signgrap wrote:
It was said that the sensor records information in a linear fashion. Does this mean that each section of the histogram is capable of recording the same amount of data?

If you mean the number of quantized levels in each section of the RGB histogram it depends on the tone curve (the slope of the tone curve to be specific) as was mentioned several time earlier in the thread. How much data is in a given window of EV of course also depends on the original scene...

Posted: Fri Feb 24, 2012 5:16 pm
by signgrap
Ari thanks for your response. Let me see if I got this right. Let's say I photograph a B/W image comprised of 255 steps from black to white, similar to the "monitor calibration strip" used by many photography sites. But instead of 17 steps typically used, there are 255 steps. The only thing appearing in the image are the steps, being careful to include the same amount of the starting and ending steps. So each value represented is exactly the same size. The resulting image would have 51 steps in each fifth of the histogram. If this histogram were viewed with no tone curve i.e. linear would the resulting display be a straight horizontal line with the same number of pixels in each section of the five section histogram? I realize that the camera histogram would look different since this has some type of curve applied.