Frost is saying that society dictates, though it should not, what is considered beautiful. A rose is commonly associated with beauty, and an apple or pear, a plum, could all be shifting perspectives of what kind of woman or thing could be beautiful. To Frost, the woman to whom this is addressed has always been what he considers beautiful, without the influence of society telling him that the plum, pear, or apple is superior.