The fortress was founded by Erik Axelsson Tott in 1475 under the name Sankt Olofsborg in an effort to profit from the political turmoil following Ivan III's conquest of the Novgorod Republic. It was sited in Savonia so as to lay claim to the Russian side of the border established by the Treaty of Nöteborg.
One of Tott's letters from 1477 includes a passing mention of foreign builders invited to Olofsborg, probably from Reval, where the city fortifications were being extended. It was the first Swedish castle provided with a set of thickset circular towers that could withstand cannon fire. It is not by accident that a network of lakes and waterways forms the setting for the castle, for these would seriously impede a prospective Russian offensive.
The three-towered keep was completed 1485, and the construction of the outer curtain walls with two towers was initiated immediately. They were completed 1495. The castle is roughly a truncated rhomboid with keep on the western side of the island and the curtain walls and outer bailey to east. One of the towers of the keep, St. Erik's Tower, has bad foundation and has since collapsed. One of the towers of bailey, the Thick Tower, exploded in the 18th century. A bastion has been built on its place. The castle was converted into a Vaubanesque fort in the late 19th century with bastions.