Puritan communities were extremely small and close-knit. Thus, townspeople acted as each other’s enforcers — if someone misbehaved, everyone else would know about it. Hawthorne makes this dynamic clear in the first paragraph of “The Minister’s Black Veil,” when he describes the way the sexton alerts the entire town to Hooper’s altered appearance. In Hooper’s funeral sermon, he says that God is always watching, but the truth is that the townspeople are always watching and judging their peers.
But although the people of Milford are always watching, they’re superficial in their judgments. Unlike God, they have no way of knowing the status of other people’s souls; they can only see others’ appearances and make interpretations of what’s beneath. Though Hooper’s appearance changes after he wears the veil, everything else about him is the same: he’s still pensive, still in love with his fiancée, Elizabeth, still eager to greet his congregation, etc. On paper, he delivers exactly the same Sunday sermon as usual, but his appearance leads the townspeople to perceive the sermon as much darker and more severe than his usual offering. A simple piece of clothing alters their perception of a man they’ve known for years.
Hooper’s appearance leads the town to imagine elaborate interpretations of why he chooses to wear the veil. Some think he’s losing his eyesight, some think he’s going insane, but most think that he has committed a grave sin and is afraid to show his face. Elizabeth, who’s clever enough to understand how powerful appearances can be in Milford, urges Hooper to remove the veil, lest the townspeople interpret it as a sign of his sinful behavior. Even though the townspeople are too timid to ask Hooper about his veil, or accuse him of wrongdoing, Elizabeth knows that their interpretations are dangerous by themselves. Indeed, the townspeople’s interpretation of Hooper’s appearance leads to his ostracism from Milford: because of the power of appearances and interpretations, he’s isolated almost entirely by the town.
And yet, over the years, while the people of Milford have been interpreting Hooper, Hooper has been interpreting them. On his deathbed, he comments on the townspeople’s obsession with appearances, saying that everyone in Milford wears a Black Veil. In a sense, this means that the townspeople have focused too much on interpreting his appearance of sinfulness and too little on their own souls and sins. Appearances are important in Milford, but Hawthorne shows how they can be counterproductive to true understanding, or true morality.