The first references to the chateau come from the mid-14th century, when the original fortress was owned by Lúček from Bohemia. Many Czech and Moravian noble families have lived here, for example, the Gentlemen from Kunštát, Kravař, Boskovice and Pernštejn. In 1768, Emanuel da Silva de Menezes e Castro, Count Tarouca bought the manor in Čechy pod Kosířem. The Silva Taroucas were of Iberian origin, whose ancestry could be traced back to the mythical king Alba Longa, who held ownership of the estate until WW2. The Silva Taroucas imprinted a contemporary classical form on the chateau, and it became famous by hosting one of the most important Czech artists - Josef Mánes.
In 1945, the castle was confiscated by the Czechoslovak state based on the Beneš Decrees and the chateau became an orphanage. In July 1957, a memorial hall for Josef Mánes was opened.
At present, the owner of the chateau and park is the Olomouc region; its management is entrusted to the National History and Geography Museum in Olomouc. Since 2016, the chateau has been open to the public.