Recently, I watched one of the Lost L.A. shows produced by KCET. This one was Lost-L.A.-Hills. Featured on the show was Gordon Pattison.
KCET.org/shows/lost-LA-hills
He grew up in The Castle...
KCET.org/shows/lost-LA-hills
KCET.org/shows/lost-LA-hills
...and lived there until 1964. His family owned both The Castle and The Saltbox.
At one point in the show, he traveled to the original site of The Castle, which is now the Wells Fargo building food court.
KCET.org/shows/lost-LA-hills
The interesting thing was that, although this was the original site of the house, it would have been located five or six stories above where he was standing.
That is how much of the hill had been shaved off.
If you look at these comparison photos Beaudry posted recently, you can visualize how high the old Castle stood on the hill, compared to the modern buildings.