Absolutely. I see it as they are making a statement that is their property and they reserve the right to say stop taking pics or to ask someone to leave if they feel like it. Which, whilst very annoying, is their right to do so. This is probably why, as others have mentioned, you can almost do what you like if you come across as a charming and pleasant individual. It gets you a very long way. That's always been my tactic - step 1, fly under the radar; step 2, be extremely pleasant if confronted.
Unfortunately, a great many photographers opt for the other tactic and get argumentative, bang on about 'photographers rights' (even though no such thing applies - or arguably exists), and video the encounter and claim harassment where there isn't any. It just leaves a very damaging impression in the mind of the security guard and they just end up labelling all those photographers with 'big cameras' as trouble makers to be moved on. A lot of the photographers banging on about 'photographers rights' usually aren't proper professionals and aren't reliant on maintaining a good relationship with local security and Police - yet they're the ones often causing the trouble.
I found this online earlier:
. The Police are simply doing their job, and this bus load of muppets just want to get argumentative and cry about how their NUJ card somehow enables them to bypass Police checks! They behave like toddlers. They could've just respected the Police, cooperated with the search, and they could've probably had a good chat/laugh with the coppers too - they're human after all! It's a small world who knows when you might meet them again. Again, a Police Officer doesn't need many encounters like this before they just think it's a lot easier to label all photographers as trouble and to give them all short shrift.