I have to agree, the 15-85 has proven a lot better for me on my crop cameras.
ive had 3 copies of the 24-105 over the years, all on crop cameras and all were good copies of the lens but left me feeling a bit cold.
Before i got wider than 24mm i used to dismiss those who said 24mm was too long for a wide end on a crop, but after trying a 17-50 i immediately saw what they mean.
Forget IQ and the L factor for a min, and concentrate on the shots you wont get with the 24-105L. 15mm-24mm is huge. even 15-17mm makes a difference. I live and work in central London and believe me, you will much prefer a wider lens as a walk about if you intend to shoot the surroundings.
On the long end there really isn't a lot in it IMO.
The 24-105 is better made, and f/4 is very handy to have, especially at this time of the year 9gloomey days, higher ISO needed etc). Thats where, IMO, the 24-105 beats the 15-85IS but its only one stop, it might not be a big deal.
IQ is pretty similar. Id say both will give you very good, sharp images. The 24-105 may have an edge but id say you will only notice the difference if you put the same shots side by side.
The other advantages the 15-85IS has is its smaller and lighter, and the IS is better. Not only is it one stop better (i believe) its also much quieter. This might make a difference if you intend to shoot hand held video in quiet places using the built in mic. The 24-105 makes a terrible racket on the audio track.
Ive actually got my 24-105 up for sale on here at the moment if your interested, even though ive tried to talk you out of getting one lol.