Color is a physical property because it tells something about the physical object it describes. If restated, "one property of a physical object is a color." For example, a desk is brown and sky is blue. But number "4" and freedom do not have colors because they're abstract, not physical.
The statement is not absolutely true, but works in most cases. Some physical objects do not have colors