Just thinking about this for a mo, can it have been that bright?
Sunny 16 rule says that clear bright sun is f/16 with shutter speed matching ISO. It doesn't get any brighter than that unless you're by the sea or something. Therefore if you set f/16, you can shoot at 1/100sec at ISO100. Should be fine for panning with anything except a really slow moving subject.
Polarising filters are great for all sorts of things, but of course they affect the image in various ways and the reduction in overall exposure is usually a bit less than two stops. Could be all you need though.