The history of Haapsalu (Hapsal in German) began with a fortress founded by the Bishopric of Öser-Wiek in the 13^th century and the settlement that arose around it. The settlement gained city rights in 1279. It was envisioned as a key hub for maritime trade, but as shipping developed and natural conditions changed, the plan never came to fruition.

Haapsalu port is mentioned in records for the first time in 1508 and 1518. The next mention came in the 17^th century, in customs documents. It has been conjectured that the historical port was located on a stretch of shore to the east of the Kuursaal, running to Africa Beach. As the land rose due to post-glacial rebound, the ports became displaced from the heart of the town.

Grand Holm, located to the northwest of the centre, had a sufficiently high water level to allow vessels to call there in the early 18^th century. Here the ships were loaded and unloaded. Back then, the waterways to Haapsalu were described as extremely complex as the bay was full of sandy shoals.

In the early 19^th century, Haapsalu became one of the most popular resorts in the Russian empire. In terms of efficiency and price, sea routes were the preferred way of reaching Haapsalu in the 19^th century. Until Uussadam port was established on Krimmi Holm in 1861, the large ships that served the Riga-St. Petersburg line called near Harilaid, where smaller vessels brought the passengers to the town.

Starting in 1861, Suur Holm's north-eastern shore was called Vanasadam (Old Port) because the tsarist government built on Krimmi Holm Uussadam (New Port). This was where most of the numerous resort tourists came into Haapsalu.

The north-eastern shore of Grand Holm, the former site of Vanasadam port, became the location of the Lääne-Kaluri kolkhoz during Soviet times, with a fishing port that was later expanded with the addition of a marina. Privatization and sell-offs following the restoration of independence resulted in today's port structure of Westmer Port, Westmer Marina, Suur-Holmi Port (Grand Holm Marina), and Veskiviigi Port.