The Duke lists all the obstacles that prevented him from talking to the Duchess directly about his problems with her behavior.
He claims that he doesn’t have the "skill / In speech" (35-36) to explain what he wants from her – but his skillful rhetoric in the rest of the poem suggests otherwise.
He also suggests that she might have resisted being "lessoned" (40), that is, taught a lesson by him, if she had "made excuse" (41) for her behavior instead.
But even if he were a skilled speaker, and even if she didn’t argue, he says he still wouldn’t talk to her about it.
Why? Because he thinks that bringing it up at all would be "stooping" to her level, and he refuses to do that.