I regularly observe in an isolated dark site, on my own, from dusk till dawn. I've been going to the same spot for eight years. It's around a mile from the nearest road, across fields with the grass cut low to indicate the path (National Trust land).
I pack my telescope (5 inch Intes Mak) , mount, eyepieces, food, water, etc into a 65 litre backpack, carry the tripod in it's case, and set off with a headtorch.
Once the sun's gone down I've never seen a soul. I tend to think who in their right mind is going to come along to this isolated spot, on a very cold night, unless they have a legitimate interest such as mine, being out in nature and enjoying the star filled sky.
I've jumped out of my skin on a few occasions, usually because a strap has caught the wind and slapped onto my jacket making an abrupt sharp sound. That, and thinking about ghost stories is about as scary as its been.
The best places to go are well away from roads, housing, people. The ones who might rob you wont stray far from their source of income.
I realise I'm fortunate enough in that I'm still fit and able to go where I go, I'm 60 and will do it for as long as I can. The sense of freedom and amazing experiences I've had observing make it all worthwhile.