Another book I've read over and over is Five Points by Tyler Anbinder. (As always, there are no Amazon referral fees when I link to a book over there. If I shill for a book it means that I actually read it.)
It's a grim (an understatement) historical account of life in Five Points, a notorious 19th century slum on Manhattan's Lower East Side. Lots of us have ancestors who lived there, and what a shitty life they led. On the other hand it was better than what they had left behind. If you feel that your life as an American isn't fair, read this and get some perspective. The Wikipedia entry for Five Points has a good but brief summary of life in this hellhole.
Last time I was on vacation I went to Manhattan and walked throughout these neighborhoods. It's all Chinatown now, with a fleeting glimpse of what was once Little Italy. (Is that right? I am going from memory 3 years ago.) Regardless, Five Points is no longer a threatening place.