We can distinguish between 'us' and 'them' by the fact that 'we' know what this field/ lane/house is called, and what happened here in former times. It makes the village postman's task more than normally difficult, however, unless he is, of course, a native himself