I have an Olympus E5 and a D800, I haven't used the E3, I think they are similar ergonomically.
Tbh I get super sharp pics on the E5 and don't find any issues with focussing, I only really use centre spot focussing though. The Olympus 50-200 and 12-60 that I use it with are, imo, fantastic. Depending on what lenses you own I am not sure that the change will give you what you are after, especially given the cost of the D800.
The D800 is still quite new to me, I do love it but with some caveats. The 36 mp sensor is brilliant, the dynamic range is far better than the Olympus. Used with a 50mm f 1.4g it is quite fast and accurate to focus although the Oly feels as good with the two lenses mentioned. I do agree with what the reviews say, the D800 will show up any flaws in lens or shooting style when viewed at 100% on these monster files.
I am keeping my E5 for zoom lens work as I doubt I will ever be able to better the quality of those lenses on a full frame body, in full frame terms they give me 24-560mm coverage with a x1.4 teleconverter.
The limit for the E5, for me is the low mega-pixel count making cropping difficult and low light performace, it really isn't good above iso 400, another area where the D800 shines.
In summary outdoors, good light, various subjects the Oly with the two mentioned lenses is superb.
Low light, cropping, big prints and detailed landscapes with tripod etc. the D800 is superb.
If I could only have one camera it probably wouldn't be the D800 given the cost of it and the lenses I would want to cover everything I like to shoot.
I am praying for a new Olympus body that will take these lenses, keep the oly colours, focus properly with them and up the mp to around 18mp.
I think what I am saying is it depends on your lenses existing and budget for new ones.