What inspired me :
Harbors were often vital for many castles, both as a means of defence as a means of trading.
The harbour was often defended by towers on higher places, like cliffs or hills right next to the shore lines.
I personally like high towers, overlooking vast oceans
Our harbor is well located for ships sailing along the sea coast, made of sturdy stone and quarried right on the isle.
Cork Ireland is an example of stone docks surrounding with one of the world's largest natural harbors.
Originally settled by monks in the 6th century, Cork -- "Cobh" in Gaelic -- was an early urban trading center after the Norse set up a trading port and the harbour has been a working port, and a strategic defensive hub, for centuries
In my imagination the Ynys docks would have been the start of the town, before all the buildings and business moved in, when the isle was a simple community of people bartering and trading for goods and services.