The first written mention of the castle dates from 1327, when it was the royal estate of Charles Robert. Later it became the property of the Németújvár family, then Sándor Köcski got it back for the king. In 1390, King Sigismund of Luxembourg donated it to Archbishop János Kanizsai of Esztergom, who belonged to the inner court. Due to the infidelity of the Kanizsa family, it became the property of the Garai family and Pipó Ozorai for several years. From 1424 it was given to the Kanizsai estate again, then to the husband of Orsolya Kanizsai, Tamás Nádasdy. By the end of the 15th century, the Kanizsaians had built it into a castle with a closed courtyard, fortified with palace wings and fortified with fortifications.