Gradually, the village developed into the fortified castle with round and square towers, defensive walls and big and small town gates as well as the citadel. In 1535 it was bought from the counts of Pazin-Pisino by the Venetian noble family Loredans whose members, although living in Barban only temporary and represented by the captain, pulled it down and built the palace and the parish church of St. Nicholas. In front of the big town gates there is a small stony church of St. Anthony the Monk from the 15th century which interior is covered with frescoes.

The most renowned historical person of Barban is a canon Pietro Stancovich (1771-1852), the author of the book 'Biographies of distinguished Istrians' where he immortalized 478 Istrian leading figures.

Today Barban is well known for its knightly game Trka na prstenac (the race for the ring), the feast of wine and the fig festival.