As the group browsed a room with paintings by Van Gogh, security guards told them they had to leave the building because people were "complaining about their smell".
The worker with the Act for Dignity charity said: "I argued with the security man, telling him the family were all decent and properly dressed.