Having grown up in the heart of London, learned how to take pictures there, witness countless thefts (including one of: 3 leicas and 5 lenses from one VERY unfortunate American photojournalist), I have a few things to say:
Keep it over your shoulder and under your arm! Taking cameras in-and-out of bags is usually a bad idea, and is when thiefs have their best chance to grab it as the strap isn't around your neck, you are stationary and not looking around you, and your concentrating on not dropping your lovely kit on a hard surface! Rucksacks (although useful, I know) can be worse as someone opportunistic could grab a nice canon 200mm L-Lens from the rucksack while your holding the camera and it's kit lens. -And you cant chase after them right away because you have an open rucksack filled with expensive kit! -Someone could also be sneeky and undo your rucksack zip without you noticing. -Ive seen this happen umpteen times.-
I found that a small second-hand crumpler "pretty boy" messenger-style bag I had solved the issue of camera in bad weather/needing it around my chest/shoulder. -I can carry the camera around in the bag, with both the bag strap, and camera strap still over my shoulder and round my chest, I just remove the camera from the bag to take a photo if needed.
In places like london, as long as its over a shoulder and across your chest, you don't really need a bag in most places. Unless well disguised, a camera bag looks like a camera bag, and most "professional" thiefs will recognise this! All the bag is there for (in my opinion) is protection from the elements if the weather gets bad!
Having said that, there are dodgy bits of london/other cities round the world where walking round with a camera/camerabag is just a bad idea, and its a much better idea to just use a bag that is secure, but looks totally "normal" rather than a photography-specific bag! Don't draw attention to yourself or the equipment! Just stuff an old, waterproof messenger bag that has a zip with some good padding material!
I often use a North Face PVC messenger bag that is totally waterproof and the zips can be locked together with a padlock. -It saved a lot of personal items (inc. passport, camera, money) in Barcelona 2 years ago!