I agree inner city and downtown should be completely dominated by the pedestrian realm, so why not make the sidewalks wider to accommodate both restaurants and the regular walking path? Even if that means taking away a parking lane or heaven forbid a lane of traffic. I know people will talk about seasonal use, but for a lot of cities people are sitting outside for most of the year (heat lanterns) and even if there isn't the demand most sidewalks in Canada just aren't wide enough as it is. The boardwalks feel like a compromise. I dunno, maybe that's just where we're at as a country.
Here's a less than ideal example, I mean it saves money for sure. From Edmonton - as a pedestrian I have to weave out into potential oncoming traffic protected by.. plastic orange cones while those folks who pay to 'rent' out the sidewalk from a private company get to claim this public land for as long as they sit there. Now before anyone gets their knickers in a knot, I'm just
messing around with the theoretical full counter argument, lol - and I'd pay to sit in a patio like this! Just trying to show it's a compromise that maybe we don't need to make.

Extended Patio by
Mack Male, on Flickr
Here's an nice Calgary example, hey just like Europe! Obviously a wide enough sidewalk and that seating area is really wide too, most Victoria patios are about half that width.

Downtown Calgary restaurant patio by
Jon Dev, on Flickr
Back to skylines.. notice anything different from this Vancouver picture uploaded to Flickr this July? Gorgeous lighting by the way, and love the red United Way flags.

Cambie Street by
Wayne, on Flickr
The photo was taken on December 18, 2013! I'm trying to think if that view has changed much..? Based on the same narrow and viewed in view/perspective.
Here's a view of the opposite direction from December 3, 2016:

Snapchatting by
Clayton Perry, on Flickr