There is one other important factor: defamation. To accuse someone of plagiarism is defamatory. If the person accused sues for defamation, the accuser needs a defence. Several defences are possible, depending on the circumstances.
Truth. The defendant has to provide evidence that the allegation of plagiarism is correct.
Qualified privilege. The defendant is protected if the allegation was provided as part of a formal relationship, such as a teacher reporting on a student's work. Communicating outside the relationship, such as telling friends at a party or writing a letter to a newspaper, is not covered by qualified privilege.
Privilege. Statements made in court or read out in parliament are protected.
It is legally safe to accuse a person directly to their face. It only becomes defamation when you tell someone else about it.