I'd imagine it's fairly ingrained in the strap itself. Maybe try some acetone (nail polish remover), but not sure how it will affect the strap.
Safer bet (what i've seen elsewhere) is isopropyl and a lot of scrubbing. Get the worst off first with a scraper of some sort, then scrub with alcohol and an old rag, constantly changing to clean sections to avoid scrubbing it back in.
Failing that, maybe try covering it with either something you make yourself, or one of those soft seatbelt cover things and stitch it in place?