Thay Pagoda, alias Thien Phuc Tu Pagoda, was built in the 11th century during the reign of King Ly Nhan Tong. At first, it was a small pagoda managed by Priest Tu Dao Hanh. The pagoda was initially built according to Sino-Vietnamese character Tam - this character is formed by 3 hyphens parallel to each other.

The pagoda therefore consists of 3 sections: Ha Pagoda, Trung Pagoda, and Thuong Pagoda. The outer part, Ha Pagoda, is a place for offerings and ceremonies; the middle part, Trung Pagoda, is a place for worship of Buddha; and finally, the inner part is a place for worship of Priest Tu Dao Hanh. An automated sandalwood statue of Tu Dao Hanh that stands and sits is located in a red lacquered shrine trimmed with gold and covered with a curtain.

In front of the pagoda is Long Tri pond, in the middle of which is a stage called Thuy Dinh, where water puppet performances are held. Nhat Tien and Nguyet Tien Bridges, built by Doctor Phung Khac Khoan in 1602, are located on each side of the stage.

Interesting sites can be visited in the surroundings of the pagoda. For example, Phat Tich and Cac Co Caves are located not too far behind the pagoda. A hole in the dome of Cac Co Cave lets one see outside the cave.