I plug into a hub unit on my desktop PC, I've laso plugged in directly, it works either way. I've previously damaged a card that wasn't inserted correctly into the SD slot on my computer, so wherever possible I use a USB cable to retrieve RAW/JPG files, its slower but safer! - certainly in my hands!!!!
If you are regularly going away I'd recommend the Hahnel cube, it can charge two Fuji batteries and has a USB outlet for charging phone, etc, and can receive power either mains or from a car lighter socket. Its not cheap but its very useful.
Its probably more risky using the USB as they dont last forever, if the port goes youll end up with a hefty bill.