6 years of deferred maintenance during the war and burning high sulfur coal as a heating fuel will do that to a city. It's no wonder that so many old buildings were demolished in the 1950s. The ornamentation and stone construction of Victorian buildings really soaked up all that grime and coal dust, making them look especially shabby.
Toronto wasn't even all that bad compared to other cities. In Pittsburgh you could see the air from the street:
Source