I went 5DII + L glass to EM5 plus selection of primes (this is where the system shines) and zooms. Excellent move, mostly. The main reason is I simply take my camera with me more often. Eg, with my 3 year old daughter, plus buggy and bags etc there just wasn't room for a bulky rucksack as well on a day trip, but not so with the m4/3 gear, and I even carry a wider selection of lenses in a smaller lighter bag!
Big downside for me is C-AF focus tracking. My daughter never sits still. Indoors she moves a lot, outdoors she moves fast a lot. With my 5DII it was easy-peezy lazy photographer time - I just placed my centre focus point on her and held the back button AF down. Whenever she moved (even by small amounts) she'd almost always be in focus. All I had to do was worry about composing and timing my shots. Not so with the EM5. When she's moving fairly slowly its fine as single shot AF is just so quick, but anything else and the keeper rate dive bombs. I've tried different techniques and tried to re-train myself as a photographer, and that's improved things a bit. But it remains the biggest compromise in the system move, which is why I'm looking at the EM1 so closely. They're the only type of photos I feel I miss compared to my Canon kit, where as the fact I have my camera with me much more (plus the amazing IBIS) means I now get many more shots I wasn't getting before. I'm sure if I really really trained myself I could get good results (like they did in the old days before AF!), but I'm not there yet.
I could waffle much longer, but I won't. Hope that helps.