That is because they heard the request to kill humans. They would feel some unease, even though they are strangers.
However, because Returner asked for it we will drive them out of the dungeon through and through, we will strike first against the adventurers and win without killing.