I've had milk Oolong once (or twice? It may have taken a time or three to finish the sample I was sent). Though it was not a daily drinker for me, it would be nice to revisit.
The 'milk' taste is from the very smooth, if not buttery taste of the tea. Some even say they taste popcorn. I do remember somethign to do with the smell or taste of popcorn.