Definitely plan to spend some time at Wistaria. There are some good tea shops on Yongkang road too, in the shopping area there. If you're looking somewhere to drink tea and have a snack, this place is good too:http://www.runawaysquirrels.com/2010/06 ... tea-house/
Muzha was kind of a bust. It's a long cab ride, and even with someone who was willing to take us around, there really wasn't much chance to see any tea production (especially since the farmer was busy actually producing tea). There's a small museum and model tea farm, but really not that exiting. I had done some research on one of the Taiwan tea forums, and no one really had suggestions of anywhere that could be visited in the Muzha or Wenshan / Pinglin areas (though people did say that the latter has more actual tea production these days). There is the cable car -- we didn't go on it, but I think it's supposed to be pretty scenic. I think you will get as good or better Muzha TGY in the city, though.
If you get a chance, spend some time in the Beitou area for the hot springs.
There's an antique market every weekend; timing didn't work out for me to go last time. The folks at Wistaria should be able to tell you when and where it is. And don't miss the night markets and street fruit carts! Shilin is the most famous night market. Cabs are relatively cheap, traffic relatively light, and drivers not too crazy, so, compared to HK or Mainland China, it's pretty feasible to take cabs almost everywhere in the city. The metro system is great too; get the unlimited card -- it's very convenient.
We didn't get a chance to go to some more rural areas when we went, but there are some agritourism type places for tea, where you can stay at a working tea farm. I know folks who have done this as part of a tea tour, and really enjoyed the experience.
I will say this -- the days go by pretty fast.
In the city, we stayed at Hotel 73
in Zhongzheng area (really nice, modern style boutique hotel) and Shuimei
(Sweetme) in Xin Beitou.