If you're tapping into the sides of the slot, then that will be a potential weak point. M5 into a 4mm slot will give you something that has about 32.4% of the overlapped area (between screw and threaded shape in the rail) compared to a proper round tapped hole. Depending on what the rail is made of, and how thick it is, there's a risk that the threads will strip. More so with M5 - it isn't a particularly coarse thread, meaning there isn't a lot of material in the ridges and valleys to resist tightening forces.
To be honest, I wouldn't do it like that, especially if it was intended to be used by others (almost guaranteed to get stripped threads). If it was for my own use (and I have good mechanical feel and sympathy), I'd use M6 (or a 1/4" 20tpi tripod thread). The coarser thread means deeper valleys (so more material to try and rip out), and the bigger diameter means a bigger overlap for the given slot width (48%).
That's a bigger overlap of something that's 1.5x stronger than M5, meaning M6 tappings in a 4mm slot would be about 1.7x stronger than what you'd get with M5 - but note that a proper M5 hole is 3x stronger than the M5 tappings in the slot.
Like I said, it depends on what it's made of and how thick it is. If it was 3-4mm thick stainless steel, it likely wouldn't matter so much (next to impossible for ordinary mortals to strip a proper M5 hole in 3-4mm stainless using a thumb screw and human fingers, so would still need a fair old effort to do it in the slot), but if it was the same thickness in aluminium, brass, die-cast zinc or the like, I wouldn't trust M5 (and would still have misgivings about M6). If it was something for others to use, I'd do it differently.
To be honest, I wouldn't do it like that, especially if it was intended to be used by others (almost guaranteed to get stripped threads). If it was for my own use (and I have good mechanical feel and sympathy), I'd use M6 (or a 1/4" 20tpi tripod thread). The coarser thread means deeper valleys (so more material to try and rip out), and the bigger diameter means a bigger overlap for the given slot width (48%).
That's a bigger overlap of something that's 1.5x stronger than M5, meaning M6 tappings in a 4mm slot would be about 1.7x stronger than what you'd get with M5 - but note that a proper M5 hole is 3x stronger than the M5 tappings in the slot.
Like I said, it depends on what it's made of and how thick it is. If it was 3-4mm thick stainless steel, it likely wouldn't matter so much (next to impossible for ordinary mortals to strip a proper M5 hole in 3-4mm stainless using a thumb screw and human fingers, so would still need a fair old effort to do it in the slot), but if it was the same thickness in aluminium, brass, die-cast zinc or the like, I wouldn't trust M5 (and would still have misgivings about M6). If it was something for others to use, I'd do it differently.