The details about the presiding deity finds mention in Brhamananda Purana, Skanda Purana and Narasimha Purana. Sage Narada got the narration about the features of Thodatri and Sargunamangai from Shiva.As per Hindu legend, Lakshmi, the wife of Vishnu is believed to have born at this place as Srivaramangai (meaning lady) and hence the place came to be known as Varangunamangai.
As per another legend, Adisesha, the serpent performed penance at this place to seek the grace of Vishnu to lay on him. Vishnu was pleased by his devotion and rested on the serpent. Garuda, the eagle also performed penance and was granted a wish to be guarding Vaikunta, the residence of Vishnu.
As per local legend, a childless king named Kaaarya worshiped Vishnu at this place.
Vishnu directed him to Nanguneri under the earth. The king dug up the earth to see blood oozing from the place. To stop the blood, the king was asked to perform ablution on the presiding deity with oil every day. Blessed with children, the king constructed the temple.
The temple has many inscriptions, the earliest of which is from 1236 CE. The inscription is from the period of Sundara Pandya I indicates an gift from him on the occasion of Adi Puram. Another inscription from 1275 from Kulasekara I indicates the details of tax collection in the region. The inscription on the Eastern Gopuram from the period of Sundara Pandya II in 1284 indicates gift of land to the temple. The gift of Vikrama Pandya III is inscribed on the northern wall in 1299. A gift of money to sculpt the image of Sadagopar in 1305 is found in the Eastern wall in the second precinct.
From the Vijayanagara rulers, the temple came under the administration of kings of Travancore. A sanyasi named Pushpanjali was entrusted with the donations from the kings. During 1447, the temple came under the administration of Vanamamalai Mutt. The first pontiff of the temple arranged for the installation of the idol of the consort of Vanamamalai from Tirupathi.
The Namboodris were also replaced by the priests from Tirupathi during the period. During the 17th century, the then pontiff sought the help of the ruling Naik king to resolve local disputes. During 1794, the Nawab of Carnatic, Abdul Umra, endowed extensive lands to the Mutt.
The presiding deity is called Devapiran as he is believed to have married Varamangai, the daughter of the pontiff of Vanamamalai Mutt. The temple is maintained and administered by the pontiff in modern times.