As per Hindu legend, the place was once a bamboo forest. A cow at this place was pouring it milk on a mound in an elevated place. The shepherd informed the king about the event and the king dug the place with a tool called Vasi.
Blood was oozing out of the place and the king and the troops were taken aback. The enemies left a pot of snakes and a snake charmer came and rescued the king.
Shiva appeared in the dreams of the king that night and informed him that the snake charmer was none other than Shiva himself.
The king was much pleased and built a temple housing the Linga found in the place.
The tilted feature of the image in the temple is believed to have been caused by the tool Vasi.
Another legend is that once, while Vishnu and Brahma contested for superiority, Shiva appeared as a flame, and challenged them to find his source. Brahma took the form of a swan, and flew to the sky to see the top of the flame, while Vishnu became the boar Varaha, and sought its base.
The scene is called lingothbava, and is represented in the western wall at the sanctum of most Shiva temples. Neither Brahma nor Vishnu could find the source, and while Vishnu conceded his defeat, Brahma lied and said he had found the pinnacle.
In punishment, Shiva ordained that Brahma would never have temples on earth in his worship. Thazhambu, the flower which was helping Brahma, was also punished for lying about Shiva.
The flower was cursed that it won't be used during the worship of Shiva, but an exception was given during Shivaratri festival in the temple.

The main entrance as viewed from the first precinct
The original complex is believed to have been built by Cholas, while the present masonry structure was built during the 16th century. There are inscriptions from later Chola emperors like Rajaraja Chola I (985–1014), Kulothunga Chola I (1070–1120) and Rajendra Chola III (1246–1279).
The first among the five inscriptions is the one on the northern wall of the sanctum indicating an agreement between Rajadhirajadeva and the temple priest with regards to the temple service.
The second one is a broken inscription on the Western wall indicating grant of paddy to the temple by a private party during the third regnal year of Kulothunga Chola I (1070-1122 CE).
Another inscription on the same wall indicates a gift during his fourth regnal year. The third recorded inscription on the same wall is during the reign of Raja Raja Chola I (985-1014 CE).
The last inscription on the Western wall of the temple accounts the grants made for the perpetual lighting of the temple made during the reign of Sri Kulothunga Chola Deva, which could have been done during either of the Kulothunga Cholas.