I've had both, sold both and miss both. Images are virtually identical, but the XE2 has a better viewfinder and for me, slightly quicker AF.
Of course, the XPRO1 also has an optical viewfinder which is so much nicer to use than any evf.
The XPRO1 is slightly bigger and heavier, but that makes it nicer to hold. It also looks great.
It really boils down to the af speed, but neither is quick when the light drops and neither is too good for af tracking.
Either would be great for what you want, but slipping in a pocket might be difficult, depending on which lens you choose.
If you're thinking of a standard prime, you may want to look at an X100/s, instead, if you fancy a zoom, the 18-55 is superb, but not really pocketable.
So, another camera to think about! Sorry.
If it was me, and I wanted the zoom lens, I would probably go for the Xpro1.
Have you thought about the X30? It like a mini XE2 with a built in zoom, but a smaller sensor. Quick af, great image quality and pocketable
Allan