There is no question that you can open a successful, profitable and rewarding tea-based restaurant with a retail section. My caution is that the way to execute this successfully may or may not match your romantic vision.
If you're serious about this, make the investment and visit the three locations of the Samovar Tea Lounge in San Francisco. While there talk to Jessie, the founder and owner. Get a look at the kitchen and behind the scenes of the operation. Samovar IS successful. Tea retail is a very small fraction of their business. They are a very cool, eclectic restaurant with a STRONG and compelling tea theme.
Also, visit Queen Mary's Tea Room in Seattle and talk to Mary, the owner. This is another very successful tea restaurant that takes a much more traditional approach. Again, she is in the restaurant business and has a tea theme.
Here is, in a nutshell, what I've tried to communicate to people who want to combine tea and food. A restaurant is a restaurant and the keys to success are the same. A Chinese restaurant, Japanese sushi bar, American steakhouse, Spanish tapas restaurant and Brazilian steakhouse are all VERY different from the perspective of the customer. They are incredibly similar behind the scenes. The life of the proprietor, chef, server and host at each of these establishments is nearly identical. There is nothing wrong with a tea themed restaurant, but you need to understand what it means to open a restaurant. (Same goes for bistros, cafes, or whatever else you want to try and call it.)
Good luck, and hit me with any more questions you have!