As per Hindu legend, sage Vamadeva visited many temples to worship Shiva. He came to the place and was standing under an Arasu tree (peepli tree) and felt that it would be nice if Shiva appeared in the place. Pleased by the devotion of the sage, Shiva appeared in the place of the Arasu tree and came to be known as Arasaleeswarar and the place came to be known as Arasili.
As per another legend, a king named Sathyavardhan who ruled the place was a staunch devotee of Shiva. He installed a garden to garner flowers for his worship. Each day the gardener complained that someone stole the flowers of the garden. The king found that a deer was eating the flowers of the garden. Angered at the deer, the king shot an arrow at it which missed its mark but hit the Arasu tree behind.
The king found that blood was oozing out from the Arasu tree, which the king found out to be the incarnation of Shiva. The scar mark in the Lingam of the temple is believed to be the arrow hit by the king.
Most of the Chola temples built during the period of 866 and 1004 CE is associated with certain military events or political campaign. During 940 Parantaka I had frequent trouble from invading Rashtrakutas. Takkolam War, dubbed as the greatest war held in the Tamil region, was fought between Cholas and Rashtrakutas.
Parantaka's son Rajaditya was killed in the campaign and Krishna III assumed the name "conqueror of Tanjai and Kanchi".
The temple has five inscriptions from the period of the Medieval Cholas like Vikrama Chola and Kulothunga Chola.
In modern times, the temple is administered by the Hindu Religious and Charitable Endowments Department of the Government of Tamil Nadu. The bronze idols of the temple were stolen and were later restored by idol wing of Tamil Nadu Police during 2009.