Testing lens performance

[parish]

My Sigma 70-300 APO DG lens is known to be "soft" and I've been getting quite varying quality with it. I've put most of this down to my technique but today I decided to test it to try and find out how much the lens contributes to this variation.

I took a total of 48 shots of a brick wall at every aperture and 3 focal lengths, each end of the range and roughly mid-way - the camera doesn't display the FL on the screen so I had to guess using the scale on the lens. 185mm is the mid-point but they actually came out at 168mm

I took the pics from ~7m using a tripod, an infra-red remote, mirror lock-up with a 2 sec delay. The camera was my EOS 400D.

Obviously I expected some variation but was surprised about how poor the sharpness was at the wide end. OK, this is a £150 lens, not Canon L-glass, but even so I didn't expect it to be as poor as this. Is this reasonable performance for this lens?

One other interesting thing is that in some of the sequences the increase/decrease in sharpness doesn't appear to be constant - it's getting better as the aperture closes down, then it goes worse for one setting, then improves again - 168mm f/5-f/5.6-f/13 and 300mm f/8-f/9-f/10. Maybe I wasn't allowing enough time for the camera to stop moving after changing the aperture? I did leave it about 5 secs. and then there was another 2 sec. with the mirror lock-up.

One thing I noticed was that the focusing ring moved (only very slightly) between some of the shots. Why was this? Was it the A/F trying to correct the fact that the lens couldn't focus? I thought that the focussing is done before the iris closes down, in which case the conditions would be constant (for each focal length) - what do people think?

All the pics are 400x400 crops at 100% from the centre of the image.

70mm f/4 1/400

70mm f/11 1/50

70mm f/22 1/13

168mm f/5 1/400

168mm f/5.6 1/250

168mm f/13 1/125

168mm f/29 1/30

 

[parish]

300mm f/5.6 1/640

300mm f/8 1/250

300mm f/9 1/200

300mm f/10 1/160

300mm f/13 1/100

300mm f/32 1/13

 

[EdBray]

Were you using a cable release or the self timer or simply pressing the shutter release?

 

[parish]

Were you using a cable release or the self timer or simply pressing the shutter release?

Infra-red remote and mirror lock-up - so not touching the camera

 

[trinitronfan]

That lens certainly is soft. Did it look sharp in the viewfinder?

 

[parish]

That lens certainly is soft. Did it look sharp in the viewfinder?

Doesn't seem so bad around f/13 to me - or does that look soft to you?

Yes, seemed OK in the viewfinder.

 

[Canon Bob]

A few observations.....

Yes, you left it long enough to settle between shots....had you not, the wider aperture, faster shutter speeds would have suffered less from any latent vibration.

Your comment about the images looking slightly worse when stopped down 1 stop could indicate that the lens suffers a slight focus shift similar to the infamous Canon EF50/1.2L

Softness to me implies aberrations and the lens will certainly have some. However, I think these shots are more likely the result of mis-focussing by the AF...reason unknown. I'd be tempted to try a few manually focussed shots at various apertures to see if you're better than the body. I'd also stand a broom handle against the wall and see if that makes a difference when using AF....at 7m, the wall should still be well within the DOF even if the broom handle becomes the plane of focus.

Bob

 

[HoppyUK]

My Sigma 70-300 APO DG lens is known to be "soft" and I've been getting quite varying quality with it. I've put most of this down to my technique but today I decided to test it to try and find out how much the lens contributes to this variation.

I took a total of 48 shots of a brick wall at every aperture and 3 focal lengths, each end of the range and roughly mid-way - the camera doesn't display the FL on the screen so I had to guess using the scale on the lens. 185mm is the mid-point but they actually came out at 168mm

I took the pics from ~7m using a tripod, an infra-red remote, mirror lock-up with a 2 sec delay. The camera was my EOS 400D.

Obviously I expected some variation but was surprised about how poor the sharpness was at the wide end. OK, this is a £150 lens, not Canon L-glass, but even so I didn't expect it to be as poor as this. Is this reasonable performance for this lens?

One other interesting thing is that in some of the sequences the increase/decrease in sharpness doesn't appear to be constant - it's getting better as the aperture closes down, then it goes worse for one setting, then improves again - 168mm f/5-f/5.6-f/13 and 300mm f/8-f/9-f/10. Maybe I wasn't allowing enough time for the camera to stop moving after changing the aperture? I did leave it about 5 secs. and then there was another 2 sec. with the mirror lock-up.

One thing I noticed was that the focusing ring moved (only very slightly) between some of the shots. Why was this? Was it the A/F trying to correct the fact that the lens couldn't focus? I thought that the focussing is done before the iris closes down, in which case the conditions would be constant (for each focal length) - what do people think?

This bit doesn't ring true:

"...sharpness doesn't appear to be constant - it's getting better as the aperture closes down, then it goes worse for one setting, then improves again."

Basically, if this is happening then it's a focus or shake issue, probably the former. Follow Bob's advice

 

[chris321]

How much do you notice this softness in everyday use? Just wondering, as I'd imagine it's fairly rare for you to be studying a brick wall at 100% under normal circumstances! I remember someone was talking about the inaccuracies of focus test charts, and 'brick wall tests', I can't remember where it was though! So I suppose what I'm trying to say is yes the difference is very noticeable on these tests, but is it really affecting your shots?

Chris

 

[parish]

Thanks for the feedback, and apologies for being slow to reply but today has been the first opportunity I've had to revisit this.

However, I think these shots are more likely the result of mis-focussing by the AF...reason unknown.

I looked at the shots in the Canon software which shows which AF point(s) were used and curiously different shots used different combinations even though, for each FL, it was exactly the same image. It wasn't just one AF point different either; some used all points, others only one, and numerous combinations in between

I would have expected it to be consistent. Is this just a limitation of the AF system due to the 400D being a low-end dSLR?

Knowing this now, maybe I should have set it to use just a single AF point for the test?

This bit doesn't ring true:

"...sharpness doesn't appear to be constant - it's getting better as the aperture closes down, then it goes worse for one setting, then improves again."

Basically, if this is happening then it's a focus or shake issue, probably the former. Follow Bob's advice

Having retaken some of these shots today - specifically the 300mm f/8-f/9-f/10 sequence as they showed this - I've got more consistent results. The focus progressively improves as the aperture is stopped down.

I couldn't improve things using MF either, however there is a very gentle breeze today - not much more than I can feel through the hairs on my arm - yet I noticed looking through the viewfinder (without touching the camera) that there was noticeable shake at 300mm. Seems my tripod isn't as rigid as I thought, at least with the weight of the camera and lens. It's an old aluminium Slik 35D.

How much do you notice this softness in everyday use? Just wondering, as I'd imagine it's fairly rare for you to be studying a brick wall at 100% under normal circumstances! I remember someone was talking about the inaccuracies of focus test charts, and 'brick wall tests', I can't remember where it was though! So I suppose what I'm trying to say is yes the difference is very noticeable on these tests, but is it really affecting your shots?

Chris

No, I don't make a habit of photographing brick walls but yes, I do notice it in my photographs. As I said in my OP I do put a lot of it down to my (lack of) technique/skill but I did this test to see how much the lens if contributing.

Here's a real life example.

This shot looks fine (focus-wise), even at larger sizes than this, but the 100% crop of the horse's face shows that lacks sharpness, the whiskers on it's snout for example.

The pic was taken with the camera hand-held, but resting on a fence, at 1/500, f/7.1, 214mm, ISO-200 using just the centre AF point - the centre of the pic is on the fence rail directly below the horse's nose.