Can I ask why the change ?
Warning - essay incoming. TLDR - I like smaller cameras and lenses.
I used to shoot m43, and really loved it. I liked the size and feel of everything, loved the insane IBIS and automations like focus stacking. Results were excellent and I just enjoyed the tiny, jewel-like lenses.
I sold up and went to the S5 because the camera body was similar in size (slightly bigger than e-m1 ii) and the kit lens was similar in size to the 12-40mm 2.8 (20-60mm 3.5-5.6). I wanted to enjoy the benefits of sharper, lower noise photos. Most of my photos are shot around 40-50mm or 100-300mm equiv focal length, which are attainable on bigger sensors easily enough. Sort of. Long lenses for ff are more expensive!
While the kit lens was excellent, it didn't feel quite as nice as the similar size pro lenses I was used to. I found image quality from the lens comparable - not better, but similar and very good. In low light, results on the S5 were typically the same or slightly worse than the m43 kit because the ibis was not as good so I'd need to use a higher iso. The compact lenses were fantastic (85mm and 50mm),built well, weather sealed etc but I rarely used them at f1.8 because too little was in focus. They didn't focus particularly closely either. So, I was usually shooting the 50mm and 85mm at f/2.8 or f4 for depth of field and by that point I was using much larger lenses for similar results to the m4/3 kit. Yes, the photos were a bit cleaner and a bit sharper, but I didn't enjoy using the equipment much and it was bigger - the oly 45mm 1.8 is TINY compared to the compact Panasonic 85mm 1.8, and while results were technically noticeably better on the Panasonic I wasn't taking it out as much so I wasn't taking as many photos.
On body size, yes, the S5 is pretty small. However it is amazing what a difference a mm here and there makes to comfort. The e-m1 ii is easily the most comfy camera I've held, for my (small!) hands at least. I could carry that thing on a wrist strap all day. In comparison, the S5 feels much bigger.
I kept taking the epl7, 20mm 1.7 and 40-150 r lens out in favour over the S5. As I wasn't using it much, I decided to sell it and get kit I much preferred to use, so I have an em1 iii, 40-150mm f4 pro and 45mm 1.8 on the way, and soon I'll grab the 12-40mm zoom to round it out. The 40-150 pro gets to nearly half size macro which is plenty enough for me so I don't even need to buy a macro lens anytime soon.
The final piece was long lenses. I don't use lenses beyond a 300mm equivalent often, but the 40-150mm f4 pro is *tiny* compared to, say, a full frame 70-300mm variable aperture. Yes, the full frame would be about as sharp, have a bit more bokeh and slightly cleaner results but honestly, I wouldn't want to carry it. It's funny how not compact a "compact" 70-300mm lens is - it weighs just under 800g where the olympus pro is literally half that. Yes, it's not strictly equivalent, but that only matters if I'm taking it out and taking photos.
On noise performance and technical image quality - I sometimes, for fun, use an old Sony A350. It has a 14mp CCD sensor, which is about as noisy at base iso 100 as a m43 camera at iso 1000 or 1600. I like the camera, it's a flimsy plastic toy that has screw drive af lenses from the 1980s but takes beautiful photos in good light, as you'd expect. With the new noise reduction features in lightroom I am able to get much cleaner images from that - effectively, low noise became even less of a draw toward larger sensors. Applying the same noise reduction to m43 images and I've got a stop or two better noise performance without the need for a bigger sensor.
If you want a budget ff mirrorless camera, I can recommend the S5 very highly. The lenses are relatively compact, light and weather sealed. They're modern so they're stupid sharp wide open, but they're also a fair bit bigger than the 50mm lenses of DSLR cameras - presumably the price of wide open performance.
Sorry for the long post. I feel a bit silly to be honest - I was very excited with the pana kit. It has beautiful image quality, good autofocus (even if contrast detect it never let me down), the body was comfy, the menu system was simple and easy to use. I just didn't find it much fun, lenses were much larger and more expensive and the results were not worth the effort and expense for me. Your mileage may vary!