I've never used a 24-105mm but I'd guess that at 50mm it'll be pretty free from any nasties associated with the wider end of its focal length range so I guess that the advantages the 50mm offers are compact size and weight, the ability to shoot at wider apertures than f4 and any advantage between f4 and some smaller aperture of your choosing.
So, would these advantages and/or differences be significant enough to matter to you?