Growing up in the southeast, Atlanta is a fairly young city compared to Chicago, NY or LA. This is the case with most southeast cities, which is also why they lack good grid systems when you get outside of the immediate downtown.
Atlanta was a railroad terminus, and much was destroyed in the civil war. By then many midwest and northeast cities were well established. It is not on a navigable waterway. One thing it (like many southeast cities) had was land, so sprawl happened. However, the city is split between counties and much of what is considered Atlanta is outside the city, and there isn't a lot of municipal control plus the state is pretty hostile to Atlanta because it is dominated by rural politicians.
That said, the city still has been growing rapidly. If you look at skyline shots from 10 years ago much of the city looks transformed. This is everything that was built in just the last 10 years in midtown Atlanta: