to be or look untidy and uncared for.
This is an idiomatic reference to the condition of someone's shoes, when they are very worn and need replacing or repairing.
He seemed well-educated, but looked so down at heel. His clothes were scruffy and he needed a haircut.