Subject-to-background distance is very important, as is flash-to-subject distance when combined with the controlling aperture value.
As for materials, you can get away with no background if you have enough space but I always preferred black velvet, as it sucks up light and very rarely impedes the shot, even when it's placed close to the subject