He said that because he is Prime Minister Prayut Chan-o-cha's brother, his family was being linked to controversies.
Pongphan has come under fire over her alleged improper public roles.
Pongphan, head of the Wives Association of the Office of the Permanent Secretary for Defence (WAOPSD), last Monday presided over the opening of a dyke in the northern province of Chiang Mai.
She was criticised after pictures were shared on social media of a sign bearing her image and name that was erected near the dyke. The "Mae Pongphan Development" dyke is named after her.