Not all poor people are problem people. Not everyone living in affordable housing are derelicts, druggies, pushers or thieves.
I come from very humble origins myself. My father died when i was 10, and my mother kept things going by working as a seamstress. I worked hard at university, and was very fortunate with scholarships and in getting accepted to medical school and a prestige residency. Things could have turned out very differently for me. People just need a chance.
I see no problem constructing a well monitored and well maintained affordable living building in a pre-existing neighbourhood. As long as the tenants behave themselves, there should not be any problems.