You may recall your mother or grandmother using a product called "bluing" in her laundry to make whites appear brighter. Bluing agents remove yellow light to lessen the yellow tinge, but optical brighteners act differently. These agents "absorb ultraviolet light and emit it back as visible blue light. This blue light masks any yellowing that may be present in the treated material and makes it seem brighter and whiter than it would otherwise naturally appear to the eye" (seventhgeneration.com). Your clothes are no cleaner than they would be if you used a detergent without brightening agents, but they appear to be.
Optical brighteners are not effective unless they remain on the fabric after washing. Clothes washed in detergents containing these agents will have a chemical residue left behind on the fabric. This is why line-dried clothing often feels stiff unless fluffed in the dryer. Clothing laundered in detergent without optical whiteners will feel soft right off the line.