Here's a quick drawing I made for how an atrium in the Bay would work out.
The red box is atrium. The teal lines mark the corridors, and the teal numbers mark the units. As you can see, 54 units, most of which would be 25'x30', fit on each floor. Over 4 floors, that's 214 units--in the same territory as our recent residential highrises.
If I have time, I'll make a drawing to show how great the units could be. I also took the liberty of copy and pasting some of the skylights from Portage place onto the bottom of the atrium, because The Bay might like having skylights into their second floor.
Anyway, the indigo box labelled "Phase 2" would carry the current street wall down Memorial, forever eliminating the loathsome parkade from view. The fuchsia box, "Phase 3", would be a highrise, standing on a parking and retail podium that would fill the rest of the block.