I live in the NW suburbs of atl, when not at school, and haven't ventured out very far yet. while there doesn't seem to be much in the way of places to buy loose tea, there are some rather nifty tea-rooms
Crabapple Tea Room and Bakery is re-opened and makes AWESOME desserts. They serve high-tea and such, but since it was rather hot when i went i opted for iced tea, which was pretty good. Their flavored iced teas are made by adding flavor shots to the tea it seems, not my favorite, but still worth the trip. I highly recommend their hummingbird cake if they have it. they have a loose-tea menu, i do not know where their tea is from though.
Mary Mac's Tea Room has recently been voted to have the best 'table wine of the south' in atl by AJC survey of the people. However i'd have to agree with the AJC staff in saying that the drunken palmers at South City Kitchen are amazing. Whenever i'm home i try to get one. It's the taste of summer for all those 21+.
There are a few other tearooms that i'm planning on checking out when i'm back home for the summer, and i'm hoping that some of them might shed some light on where to acquire non-teavana loose tea in the atl area. if i find anything out i'll be sure to post