Located on the steep hill, Kožljak's very name points to its inaccessible position. Its residential part used to be reached by stairs chiselled into a live rock, surrounded by walls.

The Kožljak castle was erected on the southern slopes of Mount Učka, on the steep cliff, along the old road connecting ancient Histria and Liburnia, the last in a sequence of feudal forts from Karst to eastern Istria along the steep hills of Učka. Erected on the site of the ancient Histrian structure, it was first mentioned in the written sources of 1102 as one of the castles given by Count Ulrich II Weimar-Orlamunde as a gift to the Patriarchy of Aquileia.

It was mentioned first in the written source under the Croatian name Iosilach or Giosilach. Its Germanic name is Wachsenstein, Slavic Kožljak or Kožlak. Wachsenstein in Croatian means the cliff of rescue, the cliff in which we look for refuge, while the meaning of kožl(j)ak is a goat's peak. Both names unambiguously reflect its rocky and inaccessible position. Castle lords added de Cosliaco or de Wachsenstein to their name for that very reason. Kožljak's natural shield as well as its impressive background are mountains of Učka and Ćićarija.

The castle is approached by stairs chiselled in the live rock. The top houses are the residential area surrounded by walls and two semi-circular towers. The Romanesque church of St. Hadrian lies beneath the castle. It is interesting to note that apart form the castle, the prison was also preserved.

Kožljak's destiny is similar to many other Istrian castles, frequent conflicts and various debts of its owners resulted in their constant exchange, from the Counts of Gorizia, the Venetians to the Patriarch of Aquileia and Austria, of course. Barbo, who also lived in the castle, became a protestant in the 16th century. One of the most prominent protectors of reformation in Istria was Franjo Barbo-Waxenstein. He played host to renowned preachers and writers, helping the protestants. However, the Church soon exiled all preachers and writers leaving them to the mercy of inquisition.