Certainly, the Sanskritic tradition, entailing reverence for the Sanskrit language and the scriptures that were orally recited and then written in this language, is a unifying factor.
Many Indians have some acquaintance with Sanskrit, but few speak it fluently.
In ancient days, Sanskrit was meant to be used only by gods, and learned men of the brāhmin caste (the priestly class), while ordinary people simply used everyday dialects in their common speech.
Even brahmin women were not expected to speak Sanskrit [Sanskṛt]. That is why women and servants speak Prakrit [Prakṛt] languages (common speech) in classical Hindu dramatic literature, while the male brāhmins speak in Sanskrit. Pali [Pālī] could be considered a Prakrit language