In 1829, Baltic German aristocratic brothers from the Ungern-Sternberg family built a broadcloth factory that operated in Kärdla from the following year. The wool needed for the broadcloth was mainly imported from outside the island and the factory’s products were exported. That was the reason behind the construction of Kärdla’s port in 1849: to serve the factory’s needs for goods.

As the settlement around the factory had quickly become the largest settlement on the island, passenger shipping lines started calling at the port around the turn of the century. In the 1920s and 1930s, commodities and broadcloth products continued to be shipped to and from the plant, and there were also flows of passengers and cargo such as livestock and agricultural produce.

The port was heavily damaged in 1944 by wartime military activity and a storm in 1947 finished off the remnants of the pier.

A new marina began to be constructed in Kärdla in 2013. The port was opened in May 2014 and there are 58 berths for recreational craft.