Type II diabetes mellitus (T2DM) is a fast-growing epidemic affecting people globally. Furthermore, multiple complications and comorbidities are associated with T2DM. Lifestyle modifications along with pharmacotherapy and patient education are the mainstay of therapy for patients afflicted with T2DM. Western medications are frequently associated with severe adverse drug reactions and high costs of treatment. Herbal medications have long been used in the treatment and prevention of T2DM in both traditional Chinese medicine (TCM) and traditional Indian medicine (TIM). This review examines in vivo, in vitro, and clinical evidence supporting the use of various herbs used in TCM and TIM. The problems, challenges, and opportunities for the incorporation of herbal frequently used in TCM and TIM into Western therapy are presented and discussed.