The elevation near a large field is the site of Kršan's tower, the remnant of the medieval castle. A little town, today's Kršan, developed around the tower as the centre of the municipality bearing the same name. It was mentioned for the first time in 1274 as Castrum Carsach, the tower destroyed by the army of the Count of Gorica Albert II. At that time, the Patriarch of Aquileia ruled Kršan. At the beginning of the 14th century, Henry of Pazin-Pisino became the owner of one half of Kršan. He was the descendant of the noble family Kerstlein from Pazin-Pisino after which the castle and the little town of Kršan were named. The tip of the arrow on Henry's coat of arms is still used as the coat of arms of the Municipality of Kršan.

The last owner of this beautiful and impressive castle was the noble family Tonetti from Plomin. Several other noblemen also originate from Kaštel. In the Middle Ages, they filled important posts in the Pazin-Pisino County. The best-known nobleman is Andrija of Kršan, Captain of Pazin-Pisino from 1488 to 1500. His tombstone with the family coat of arms is now situated in the chapel of St. Henry in Belaj where it had been transferred from the tomb in the church of a former Paulist monastery on the shore of the Lake of Čepić. 

In the 19th century, a transcript of Istarski razvod (The Istrian Book of Boundaries) was found in Kršan. It was made by the priest from Barban Jakov Križanić. This important historic and legal document was published in 1852 by Ivan Kukuljević Sakcinski, and it was transcribed in Latin script and prepared for print by Dr. Ante Starčević, later a prominent Croatian politician.

Only a four-sided tower has been preserved of the old castle. Later on, defensive and residential buildings were erected around the tower. You could enter the castle using a stairway located in the entryway connected with the covered courtyard with a cistern in its centre. The courtyard was later opened towards the town following the opening of a door. Together with the dominant tower, the complex of the castle in Kršan consists of several rooms for the staff and the weapons, a large building used as residence of estate owners and a large separate kitchen.