Actually London's Olympics are the second most expensive ever ($30 billion as opposed to Beijing's $44 billion).
Even then much more gets done but isn't part of the official budget.
For infrastructure alone, in the lead up to the Games London has so far completed:
1. a new $8.6 billion airport terminal doubling the size of the world busiest international airport, and largened the city's lead as the the worlds busiest airhub (140 million passengers p/a), alongside most internationally visited city.
2. Completed the $7.9 billion Channel Tunnel Rail Link to the Games site, alongside a new $1.5 billion terminal - total HSR development cost so far $10.9 billion.
3. East London's DLR (light rail) refurb and extension to increase line usage to 80 million
4. Created the new London Overground network (Games site is at the start, in Stratford):
http://bnb.bpweb.net
5. Expanded east London's Stansted Airport from 15 million capacity to 25 million this year (and given the green light for further expansion to 35 million).
6. Opened its $1.2 billion 90,000 capacity New Wembley stadium where the soccer will be held. It joined new stadiums of 60,000 (Emirates), 80,000 (Twickenham) and 80,000 (Ascot).
7. Totally rejuvenated all major streets, tube stations, parks, museums and attractions at untold cost, in hundreds of separate projects.
8. Tore down and regenerated each of the most notorious estates in the city, mostly in the east (Kidbrooke, Ferrier, Aylesbury, North Peckham), at $7.5 billion cost. The last, Elephant and Castle is currently undergoing a $2.3 billion masterplanning.
8. Built and added thousands of extra hotel rooms every year, with a total of 13,300 extra by 2012 (total number of hotel rooms by then 123,000).
^The thing is none of this counts as Olympic development.