I used to work for a photo booth company, and a lot goes into it and there is a lot of competition. However it's my opinion that open space sets are better than traditional photo booths, especially when you throw in the accessories (don't go for cheap stuff from pound shops, look online for quality stuff) such as masquerade masks, glasses, moustashes on sticks, even go for themes ie. pirates, vintage, circus, etc. Most companies include instant prints and a guest book as part of the package, so you'll need to invest in a good printer. I'm happy to provide further advice in a PM.
As a guide, most (good) photo booth companies charge £450-£600 for 3 hours, some are even higher. You could afford to charge more dependent on the backdrops, accessories and what's included in the package.