Augmented Reality (AR) has really become popular in the past few years with the emergence of smartphones which provide processing power to AR based applications. Most smart phone based AR applications are limited in user interaction they provide with most employing touch based interaction. The goal of the research is to provide support for gesture based interaction to virtual content generated in user phones. The hand detection algorithm has been designed in such a way to leverage the low processing capability of smart phones.