I guess you're talking about the 100-400, which I have too. Occasionally I get caught out, but very rarely. TBH I'm not sure there is any improvement in AF speed by using the limiter, if your focus distance is already approximately right for the subject distance and you don't screw up and lose track of the subject altogether.
If the camera/lens can't see a thing then it will rack the focus all the way to minimum distance and then back out to infinity, before giving up. By setting the focus limiter you avoid it focusing needlessly close, wasting precious milliseconds and having the scene extremely OOF. If the camera/lens can make out the subject well enough to figure out if it close or further away then the lens will go straight there, so the focus limiter has no effect in such cases.
By using back button AF it may make it a bit easier to interrupt the camera/lens' attempt at focus searching and you can re-acquire the subject and hopefully the camera/lens will lock on straight away.
I also find that since it is the system's preference to bring focus closer by default, when searching for focus, it is often helpful to "reset" the focus to a farther distance in between shots. That way it should snap into focus a little quicker, rather than shunting around deciding which way to go. It should also make it a bit easier for you to find the subject, if focus is some way off where it needs to be. Having the lens focused at 2m is no fun when you are trying to locate a subject that is 40m away.
Now, personally, I mostly leave the limiter switch at 6.5m, usually my subjects are well beyond that distance, but in theory, if I practice the techniques described above, I think I probably would be perfectly fine leaving it set to 1.8m. I very rarely see the lens racking focus all the way.