It should be remembered that Queen Victoria's father Prince Edward spent most of the 1790s in Canada, and made some notable contributions to Halifax, including commissioning, designing and building the town clock (still extant on the side of Citadel Hill) and, he also had quite an estate built on the Bedford Basin, with one building on the estate still extant (Prince's Lodge), highly visible from the Bedford Highway. He was a paragon of the local social scene at the time, and kept a high profile mistress in the colonial town.
Maybe Harry could do something similar (sans the mistress of course).