I don't see why there's a big issue here. The old building fronts Mill St. and needs to stay, the rest of the property isn't remarkable and could become something great. Maybe not 8 storeys of it, but 5 or 6? Even three, taking up more of the area, would be ok IMO.
Change is a scary thing, I guess.
