The Duke explains to the listener why he brought up the painter, Frà Pandolf.
He says that he mentioned Pandolf on purpose, or "by design" (6) because strangers never examine the Duchess's portrait without looking like they want to ask the Duke how the painter put so much "depth and passion" (8) into the expression on the Duchess's face, or "countenance" (7).
They don’t actually ask, because they don’t dare, but the Duke thinks he can tell that they want to.
Parenthetically, the Duke mentions that he’s always the one there to answer this question because nobody else is allowed to draw back the curtain that hangs over the portrait.
Only the Duke is allowed to look at it or show it to anyone else. This is clearly his private gallery, and we’re a little afraid of what might happen to someone who broke the rules there.