As per Hindu legend, sage Garga performed his penance at Sri Karunakara Perumal temple and obtained knowledge. The place thus derived its name Garagaham, which later became Kaaragam.The Moolavar of this Sthalam is Sri Karunakara Perumal. He is found in standing position facing his thirumugham towards South direction (direction that belongs to the cool breeze).
The Perumal in this sthalam is Karunakara Perumal. He is giving his seva on facing South direction, which is said to be the direction that belongs to the cool breeze. He is found along with Padmamani Naachiyaar also known as Ramamani Thayar and explaining to the world that he is the Megham (Clouds) which brings rain to this world to enrich the wealth of the world. Since he shows Karunai (Courtesy), the Perumal is called “Karunakara Perumal”.
The temple is revered by the verses of Tirumangai Alvar.