Well, Whilst I haven't replaced my Nikon full frame kit with M4/3, I've added it to compliment the Nikon's (at least that was the plan). Purchased a GX7, then a OMD-EM1 (with the free Olympus promotion Grip), and a few nice lenses (Panny 7-14 F4, Oly 12+40 F2.8 & Panny 35-100 F2.8 to name but three), and since getting into M4/3, I haven't used my Nikon kit once.
Does that mean it's better than or as good as a decent DSLR, well yes and no. For me at least, so long as the ISO is kept to 3200 or less, I'm not finding any real issues with noise, that a decent NR program can't handle, and certainly up to ISO 800-1600, very little if any noise reduction needs to be done. The burst rate, autofocus, build and specs of the EM1 (and GX7 for that matter) are superb, and easily a match for a mid or upper end APC-C DSLR. I haven't yet found a situation where I'd wished for my D800 or D3s. The Panny and Olympus lenses are absolute belters, and combined with the three I mentioned, the Olympus 45mm F1.8, 75mm F1.8 & 60mm F2.8 Macro, as well as the Panny 20 F1.7, 14mm F2.5 & The Panny / Leica 25mm F1.4 are all superb (I have them all except the 75mm and 25mm Leica.
What takes you by surprise no matter how many times you might hear it or read it, is how small and light a decent M4/3 kit is. My gripped OMD-EM1 with 7-14, 12-40, 35-100 & 100-300 kit weighs just over 2kg, where as my Gripped D800 with 14-24, 24-70, 70-200 and 80-400 VR weighs in at over 6kg, and is about 4 times the size.
Where the DSLR's's do beat M4'3 in my opinion is in really low light (above 3200 ISO) as sheer physics dictate that the larger sensor sizes mean better low light performance (although until you reach full frame, the difference isn't quite as big as you might think). Also, continuous AF tracking on the best Canon & Nikon DSLR's still beat in my opinion the best that M43 has to offer. One other main area I think where the big boys beat Olympus and Panasonic is long lenses. Currently in the line up there's a 300 F2.8 from Olympus's 4/3 range (which can be used with a appropriate adapter), but it's rare as hens' teeth and a kings ransom in price, or a 50-200 (100-400 effective) Olympus 43 unit. Panasonic has only really the 100-300 (which I have), and in good light it's great, but in poor light, it's F5.6 aperture coupled with M4/3 average very high ISO performance, means that images can get noisy. Both Nikon & Canon have decent F4 telephoto units (300mm) and seriously good 300, 400 & 500mm F2.8 & F4 primes, all of which can be had less than the Olympus 300mm F2.8 new. To be fair, Olympus have announced a Pro 300mm F4 (600mm effective) for later this year, but no announcement on price.
the other thing which may affect some people is resolution. Whilst the image quality from M4/3 at low to medium ISO wrings the absolute best out of the sensor and quite often belies it's size, M4'/3 currently tops at 16mp (4:3 aspect ratio) and if like me you like to shoot in 3:2 aspect, that drops to 14mp. Now whilst I've still produced lovely landscapes with the EM1 and GX7, which would blow up to A3 and well beyond, there's no getting away from the fact that a 24mp or 36mp sensor (as my D800) captures an astonishing amount of details that the M4/3 simply can't (namely leaf and grass detail) in landscape scenes. But here's the rub, to get the very best out of these cameras (specially my D800), I have to use top of the line lenses (24-70, 14-24 etc), which are very pricey, as well as top notch camera techniques (tripod, mirror lock-up, cable release etc), where as the EM1 especially with it's brilliant Image Stabilisation, quite often allows me to shoot hand held without worries.
So in summary, unless you are a birder, shooting fast sports etc, I don't really thing M4'3 will hold your photography back, indeed quite the opposite as I've found myself always carrying an M4/3 body and a couple of lenses with me all the time, as they are so small and light, something I couldn't really say with my Nikon kit.
Obviously the above are purely my views and YMMV of course.