As an owner of a 17-40mm, I found it too restrictive on its own. I couldn't afford to have both so the 24-105 was more of a sensible choice for me. It was fun to use though, I must admit!
If money isn't an issue, I'd strive to own both, you already have the 24-105 so with this and the 17-40mm...