Come on then Hoppy, that link I posted explained exactly why it is NOT technically flawed, the least you can do is elaborate on why you think it is, being the standard method of testing as well.
Most people have already said it already, and yes, the author does go on at some length to justify his test against the waves of protest he's received. And of course he has a position to protect
I know that it drives them mad in camera service departments when they get lenses returned for calibration, which are within tolerance and perfectly acceptable in normal use, and then the customer is upset as they get a bill for nothing plus a load of frustration and aggro.
The problems I have with it are because it is shot at 45 degrees (not square on) and at very close distance.
TBH, I don't have a big problem with it being at 45 degs. Not ideal, and it's only done so that you can read off the distance directly very easily, so that is a poor compromise for a start. The theory is that AF points are active over a much larger area than appears in the viewinder, and a larger area than the focus line on the target. There is also some mumbling about the orientation of some focus arrays, but I'm not sure it makes any difference in practise. I haven't found it so, but then I've only checked a couple of different cameras, both Canons.
Now the close distance. Unless you have a macro lens, why would you want to shoot that close? More importantly, lenses are analogue devices - they are not perfectly calibrated to meticulous accuracy over a vast range, they are calibrated to optimum accuracy over more common distances, and they are also calibrated within a tolerance. When the lens gets within that tolerance, it doesn't try any harder to nail perfect focus; if it did, it would hunt and shunt about and be very much slower. Lenses are also often different in this respect, depending on maximum aperture and how much fine movement they have in the focusing helicoids at close range. Only macros are built with this kind of fine accuracy in mind.
Zooms are subject to tolerance over their focal length range, and if you self-calibrate a zoom to focus perfectly at one focal legnth and very close distance, it will quite likely be out both at longer distances, where you actually need it to be spot on, and also at different focal lengths. The web has plenty of examples of this happening, and I've read it on here a few times. Again, all lenses are different in this respect and I guess it is reasonale to assume that a Canon L will be better overall than a cheap mega zoom, but the chart takes no account of that.
User error. When you look at the chart, depth of field is only a few mm. In practice, it is extremely difficult to work to that level of accuracy. Don't ask me why, if you're careful and know what you're doing, but feedback from folks who have done that test, and been disappointed, shows that more often than not they have got it wrong and actually the kit is fine. StewartR can vouch for this, as I guess a lot of people will check out his hire lenses with that chart and tell him they are duff, when they are obviously not.
But my question would simply be, why use a small chart like that, which is so prone to errors, variables and irrelevances, when all you need to do a proper test is three cerial packets and a ruler in the kitchen? The benefit of that test (intellectual copyright HoppyUK
) is primarily in that it is shot at a reasonable distance - about the same range as you might shoot a portrait, and therefore within the range that you want, and would also reasonably expect the lens to handle perfectly. The other thing is that the target is flat, and much larger than the AF point, with lots of easy detail for the system to lock on to. Even so, I would not use the cerials packets test, as I've described it here, for wide angles. You would have to move too close and field curvature at least is a concern. It could be modified though.
I will try to find a link to the instructions that Canon issues for self-calibration of lenses, which are broadly similar to the method I've described. The key aspect that they stipulate is, I think, that any tests should be done from a distance of at least 50x focal length.