The one who said it in a aggravated tone was Luke who had remained silent until now.
"Men occasionally have times when there's nothing they can do about it. It's not like Rudeus didn't embrace Sylphy because he doesn't like her. Rather, in my opinion, I can finally comprehend the reason why he was so quiet up until now."