As a ROUGH rule of thumb, if the shutter speed is slower than the reciprocal of the focal length of the lens*, I'll use some sort of support - either bracing against a wall or using a beanbag, monopod or tripod. On a good day, I can get 1/2 a stop slower but on a bad day, I need VR for 1/1000th!!!
*ie, if I'm using a 60mm lens, I want to try to get the shutter speed up to 1/60th of a second and if I'm using a 500mm, I'll want 1/500th. This is only a rough rule of thumb - practice and experiment for yourself to see what you can handhold with your lenses.