The Great 400mm Canon Shootout

Interesting compare.

Does the smooth background blur possible and higher shutter speeds
with a 300 2.8 trump the additional detail in 400 5.6?

Jack_IS wrote:Does the smooth background blur possible and higher shutter speeds
with a 300 2.8 trump the additional detail in 400 5.6?

There's just no possible way to answer that Jack. It totally depends on the subject and photographers vision for the shot (i.e. when, where, and how the lens is used).

Even if you want to phrase it as "is the extra cost of the 300/2.8 worth it for the extra 2 stops", it would be a total value opinion, and nothing objective. In both cases you're getting a great lens, so these are all minor trade-offs, nothing that say "lens A is clearly superior to lens B".

E.J. Peiker wrote:

liquidstone wrote:So..... the 100-400 IS vs 400 5.6L debate rages on....

I actually don't think its much of a debate

On the one hand we have:

1) Liquidstone's zoom outperforms the 400 prime.
2) The http://www.photozone.de/ site tested both and shows the zoom beats the prime.
3) Canon's lens MTF charts show that, by design, the zoom is intended to beat the prime

And your test which shows the opposite.

If that isn't room for a debate I don't know what is...

bjs wrote:

E.J. Peiker wrote:

liquidstone wrote:So..... the 100-400 IS vs 400 5.6L debate rages on....

I actually don't think its much of a debate

On the one hand we have:

1) Liquidstone's zoom outperforms the 400 prime.
2) The http://www.photozone.de/ site tested both and shows the zoom beats the prime.
3) Canon's lens MTF charts show that, by design, the zoom is intended to beat the prime

And your test which shows the opposite.

If that isn't room for a debate I don't know what is...

I am certainly no expert, but it only makes sense to me that examples of the zoom beating the prime are anecdotal and an adequate sample size IMO would show that the prime is generally sharper than the zoom. That doesn't change the fact though that the zoom is a very handy lens and is capable of producing excellent image quality. But I still believe the prime is generally sharper than the zoom. Again, my opinion.

1) Liquidstone's zoom outperforms the 400 prime.
I don't doubt that in his case it does however I would not expect this to be the norm

2) The http://www.photozone.de/ site tested both and shows the zoom beats the prime.
The only two tests that are comparable to what I did is the MTF50 at 400mm test on the center pont in which the 400/5.6 slightly outperforms the 100-400 at f/5.6 Furthermore it dramatically outperforms the 100-400 on the edges (which in these tests is a 1.6 crop frame edge which arguably makes it not an edge test at all since it says nothing about how either lens will perform with a 1.3x or full frame camera - the fatal flaw of the Photozone.de reviews and the reason they are completely invalid unless you will only ever shoot with a 1.6x crop camera - of course mine doesn't show the edge either but mine is not and was never intended to be a comprehensive test of all lens parameters)

3) Canon's lens MTF charts show that, by design, the zoom is intended to beat the prime
Not quite sure what MTF charts you are looking at but the ones posted on the Canon site do not corroberate that statement.

Used 30d, 100-400L indoors on tripod, no IS, about 15'. Timer. 1/250ss with studio strobes. SHot in RAW, processed in C1LE.

On left are without any sharpening at all during RAW conversion.

On right with default C1LE sharpening.

No other pp of any kind.
Wish I had my 400mm f5.6 but I sold it about a month ago.

Patrick Cox wrote:I am certainly no expert, but it only makes sense to me that examples of the zoom beating the prime are anecdotal

Anecdotal? I guess this just shows how strongly people filter out data that doesn't fit their preconceptions....or perhaps you didn't actually read item #2 in my post that you quoted...or maybe your view photozones tests as anecdotal in which case you have a very unique definition of anecdotal.

bjs wrote:

Patrick Cox wrote:I am certainly no expert, but it only makes sense to me that examples of the zoom beating the prime are anecdotal

Anecdotal? I guess this just shows how strongly people filter out data that doesn't fit their preconceptions....or perhaps you didn't actually read item #2 in my post that you quoted...or maybe your view photozones tests as anecdotal in which case you have a very unique definition of anecdotal.

Look, I have owned both lenses and I chose the zoom because I liked the range and the IS, so I certainly don't have any bias towards the prime.

Regarding photozone, while I believe their test was valid, it is still "anecdotal evidence." I would not base a theory on one test comparing one copy each of two different lenses. Additionally, it sounds like they compared two copies of the zoom and chose the best but they only tested one copy of the prime. And they even say in the review that it is a "Very good performer...assuming you get a good sample."

Finally, I am not sure you have interpreted the Canon MTF charts correctly.

Again, I don't really care which lens is sharper, but in all of the info I have gathered, I am of the opinion that the prime is generally sharper.

Regards,
Pat

Patrick, Let me get this straight, you're saying that actual test data is ... anecdotal?

ColorChange wrote:Patrick, Let me get this straight, you're saying that actual test data is ... anecdotal?

My point is that if you test one sample of a lens, that is not a large enough sample size to make an assertion about the performance of that lens vs another. Therefore I have called this "anecdotal." If I am using this term incorrectly, then my error.

OK, cool. Anecdotal evidence is a bunch of clueless people on the internet expressing opinions (what you get 98$ of the time). EJ's presentation of testing data is not anecdotal. Actual testing data is the opposite of anecdotal evidence. Unfortunately, and the point you are getting at, that isn't the end of the story.

What you are describing is statistical significance. There is lens to lens variation (usually anecdotal actually ), in addition to testing variation (error). If the lens-to-lens variation is comparable to the resolution and repeatability of the test measurements, the conclusion is that the results are statistically insignificant. On the other hand, if the lens-to-lens variation is much larger than the testing variation and resolution, then the results are statistically significant. The only way to know you are reaching a legitimate conclusion is through ad ANOVA analysis (analysis of variation).

Regarding EJ's work, he could have had unrepresentative (good or bad) lens examples. Because he has only one lens of each to work with, and because I have never seen an ANOVA analysis for lenses, we have to be careful concluding anything from on set of tests. Now, if we quite a few people getting similar results we can start to believe the results are statistically significant.

Also regarding EJ's testing, his testing method could be fraught with errors that could lead to incorrect conclusions. This is unlikely because EJ is a highly trained engineer that works for a company that lives and dies by statistical analysis and process control so I am sure he is intimately familiar with the topics I have glossed over here. I would be willing to bet his testing methodology was quite accurate within the constraints he described.

So, can we conclude with confidence anything from EJ's test alone? Technically speaking ... no. But because his testing is consistent with other testing, I think you can accept his general claims.

ColorChange wrote:OK, cool. Anecdotal evidence is a bunch of clueless people on the internet expressing opinions (what you get 98$ of the time). EJ's presentation of testing data is not anecdotal. Actual testing data is the opposite of anecdotal evidence. Unfortunately, and the point you are getting at, that isn't the end of the story.

What you are describing is statistical significance. There is lens to lens variation (usually anecdotal actually ), in addition to testing variation (error). If the lens-to-lens variation is comparable to the resolution and repeatability of the test measurements, the conclusion is that the results are statistically insignificant. On the other hand, if the lens-to-lens variation is much larger than the testing variation and resolution, then the results are statistically significant. The only way to know you are reaching a legitimate conclusion is through ad ANOVA analysis (analysis of variation).

Regarding EJ's work, he could have had unrepresentative (good or bad) lens examples. Because he has only one lens of each to work with, and because I have never seen an ANOVA analysis for lenses, we have to be careful concluding anything from on set of tests. Now, if we quite a few people getting similar results we can start to believe the results are statistically significant.

Also regarding EJ's testing, his testing method could be fraught with errors that could lead to incorrect conclusions. This is unlikely because EJ is a highly trained engineer that works for a company that lives and dies by statistical analysis and process control so I am sure he is intimately familiar with the topics I have glossed over here. I would be willing to bet his testing methodology was quite accurate within the constraints he described.

So, can we conclude with confidence anything from EJ's test alone? Technically speaking ... no. But because his testing is consistent with other testing, I think you can accept his general claims.

Thanks for this info Tim. I understood that "anecdotal" basically referred to people's accounts of their "experiences" but I also thought it could mean real test results but of only one sample. Kind of like one person's testimony in a legal case might be considered anecdotal. But I guess this use "of one sample" does not apply in this context. So my bad.

Regarding EJs test, I believe that EJ's test is valid and reflects reality. In all of the evidence that I have seen, anecdotal and scientific, I have personally concluded the same as EJs test.

Thanks again,
Pat

EJ;

Would you please send the $5 bill that you used to my home address so I can do a similar test???

Hugh

mrhughj wrote:EJ;

Would you please send the $5 bill that you used to my home address so I can do a similar test???

Hugh

A single sample would only be anecdotal, you should really request one from each participant of this thread, to get a good statistically significant sampling.

Patrick Cox wrote:I understood that "anecdotal" basically referred to people's accounts of their "experiences" but I also thought it could mean real test results but of only one sample.

Pat: your definition is correct. As per Merriam-Webster:

based on or consisting of reports or observations of usually unscientific observers.

Knowing E.J.'s meticulous attention to detail, I would not call his observations anectodal. They are, however, coming from a small sample and subject to variance from one lens to the other, as he was careful to point out.

I'm much more inclined to believe E.J.'s report than Photozone's.

Rob

I believe this has been mentioned earlier in this thread but FWIW, there is one other test comparing the 400/5.6 to the 100-400 that was posted over on Luminous Landscape - http://www.luminous-landscape.com/revie ... -400.shtml

And.... very anecdotally both EJ's and the Luminous Landscape tests mirror my own experience with these lenses. I used to own the 100-400 and was able to get some very nice sharp images, but when I purchased the 400/5.6 I noted that the sharpness of the images was much more consistent and the images held up to cropping much better with the 400/5.6 than the 100-400.... anecdotally.. ... I sold my 100-400 a long time ago.

I'm sorry I got this anecdotal thing started! I won't let it happen again!

E.J. Peiker wrote: 3) Canon's lens MTF charts show that, by design, the zoom is intended to beat the prime
Not quite sure what MTF charts you are looking at but the ones posted on the Canon site do not corroberate that statement.

Just to clarify which MTFs, here are links to the charts for both lenses from Canon-USA site. The MTF charts appearing in my Lens Work III are exactly the same.

400 5.6
http://www.usa.canon.com/consumer/contr ... delid=7314

100-400
http://www.usa.canon.com/consumer/contr ... delid=7344


At 400 mm wide open and at the center of the frame (I understand EJ's crops are central too), here are the approximate MTF characteristics from my reading of the charts:

30 LPMM:

100-400 - 0.90

400 5.6L - 0.76 to 0.77


10 LPMM:

100-400 - 0.98 to 0.99

400 5.6L - 0.91


If I read the chart wrongly, kindly correct me.


Romy

Just for the reference, here are the MTFs that i copied from Lens Work III ebook.

So from the MTF charts, the 100-400 is better in the center of the frame than the f5.6 prime AND the 300 +1.4x ???