I agree that, if you are planning an enclosure, it is best to wait till then. If they've been allowed to roam first, they may feel restricted by the enclosure. Whereas if that is all they have known, then they will find it wonderful.
I let our two cats outside from first thing until dusk. More cat fights occur at night and there is wildlife here in Australia which is a danger to the cats - and vice versa. We have a large garden and are 40 metres from the road. Of course, there are risks, so it is a very individual decision. In part, our decision is based on our lifestyle. For the warmest 6 months of the year, we open the house up first thing, eat all meals outside in the garden, and don't come indoors ourselves until it's dark. If the cats weren't allowed out we'd never see them! Also, we are lucky not just with our position, but because I work from home. So I call them if I haven't seen them for a while, to make sure they are ok. They come when called.
To answer your question as to trials and tribulations with neighbours, we have had a few. On both sides of us are couples where the woman loves the cats, gives them cuddles and is happy to see them in their garden. Their husbands are ok with that, sort of, but don't like it when one of the cats goes into their house! (Australia + summer = houses open all day......). I have apologised and explained that, as the ladies like them, the cats feel confident there and the only way to stop it is if everyone ignores them instead of encouraging them. They all have my mobile number and I assured them I would come straightaway and take the cats home if it happens again.
Our cats have tags with their names and my mobile number. We were called by a house three doors away to say Jonesy was in their garden. They weren't bothered, but thought we would be worried. We collected him. The second time they called, I suggested I just call him to come home, and they were fine for us to try that. I called him and he came back to me, over and under the fences.
So, no real harm done. It helped that we visited all our close neighbours to say we had new kittens and let them know once they were starting to go outside. No one will be happy if they kill any birds or other wildlife (including us!) but that is also one of the risks we are taking and I guess I will need to apologise if one of them kills a bird in their garden
It's a conundrum. Personally, I wouldn't have it any other way. The cats adore being outside, especially when I call them up to our big veggie patch. They really like it when I am out and working in the garden. Really big playtime!!!
All the best