If you commit damage you may well be convicted of that.
Trespass is a civil matter in a lot of cases, however it does become criminal in certain cases when dealing with national infrastructure (telephone exchanges, nuclear plants, military bases, ports (including air and sea) etc