I've seen this question asked a few times, and as ever it depends on what you shoot.
The Canon 70-200 f/4 is widely regarded as just about the sharpest zoom that canon make, reasonably priced, and relatively small and light.
The sigma is bigger, heavier, slightly less sharp, but an absolute steal at the price.
However, the big question comes down to whether or not you need the f/2.8 aperture. If you shoot in low light situations, and you can't make use of any of the canon's plus points, then you'll have a blurry mess. If you don't need the f/2.8, then the canon will do a stellar job and be easier to carry around as well. They're two very different animals!
Chris