The fort is quite ancient. Remnants of Microlithic man have been discovered here. The various Puranas (ancient scriptures) like Matsyapurana, Agnipurana and Skandapurana include many references about Harishchandragad. Its origin is said to have been in the 6th century, during the rule of Kalachuri dynasty. The citadel was built during this era. The various caves probably have been carved out in the 11th century. In these caves are idols of Lord Vishnu. Though the cliffs are named Taramati and Rohidas, they are not related to Ayodhya. Great sage Changdev (one who created the epic Tatvasaar), used to meditate here in the 14th century. The caves are from the same period. The various constructions on the fort and those existing the surrounding region point to the existence of diverse cultures here. The carvings on the temples of Nageshwar (in Khireshwar village), in the Harishchandreshwar temple and in the cave of Kedareshwar indicate that the fort belongs to the medieval period, since it is related to mahadeva as a totem of tribes Mahadev Koli. They were controlling the fort before Moguls. Later the fort was under the control of Moguls. The Marathas captured it in 1747.