I have to agree. The interesting Victorian-ish storefront elements are long gone, and the building doesn't hold much architectural value in its current form (IMHO), even if it is a legitimate heritage structure. I wouldn't be opposed to a redevelopment, but as you say it would have to be high quality for me to be on board. If somebody could do new construction with authentic heritage cues (not the typical 'faux hertiage' stuff we tend to see) it would help to cement the feel of the area.
Although then, there would be the obvious optics of tearing down a legitimate heritage building to build a 'fake' one...
I think it would be a reasonable compromise, though, if it were done right, added density, and improved the street feel of the area. If the empty lot next door could be part of a new development, this would be an additional improvement.