Just happened to see this picture and I thought it nicely illustrated how much potential this spot has but how it's in a compromised state right now, held back by a couple of problems:

Source
Too bad the old Royal Bank building was torn down. The old Bank of Montreal looked interesting too. Not very large but with a grand entrance much more impressive than what the modern office tower has.