@bolognium, not calling you a NIMBY, I've read your comments and you certainly are looking at the development from a rational basis, than an emotional (NIMBY) one in balancing the competing interests.
It was the comments that are in the city report on the development. Some near residents/neighbours of the development are of course opposed to it for the usual NIMBY crap; "Their view" will be obstructed, the increase supply new of condos will devalue theirs, they assumed the homes would remain for time-immemorial. IMO they realized that those opinions hold zero sway to stopping the development, so they piggy back on the heritage issue in hoping to stop or delay the development with that argument.
This is a London wide issue though. Nearly every proposed development to in-fill the core seems be getting detractors for some, marginal at best, NIMBY arguments. Wellington & Wolf development because their backyards will be seen into, Clarence & King the design is weird/awkward, Middlesex Health building redevelopment, people from the Renaissance towers assuming they "own" their view to the river wanting to veto it. I'm sure the mentioned Rygar development will face similar arguments.
London needs investment in its core, and if developers are willing to invest $100s of millions to get it out of mediocrity why stand in the way!? Heritage and history is important, but should not trump future development and prosperity. Talbot Block a perfect example, while it was historically significant, and was the heart of old London, its redevelopment into Bud Gardens helped transform the area and give it economic growth much more than the old block could've done.
On top of that, if RT & High Speed Rail come to the core in the next 10 years it will make the core a more desirable place to live, get more economic growth than can be imagined now!