I would not add milk or sugar to Oolong, or any tea other than fully fermented tea.
The tea called Black Tea in the west, or Red Tea in China can be taken with milk and sugar. If you have a cheap commercial Black Tea, and YOU like it with Milk and/or Sugar that is great. The finer (and more expensive) your tea becomes then, less and less will it be a suitable vehicle for milk and sugar for they will both overpower the fine drinking qualities of the leaf that you are paying for.
Just a little step up from your cheap commercial Black Tea and I would advise cutting down on your Milk and Sugar. Add Milk and Sugar to a First Flush Single Estate Darjeeling and I suspect there will be a Lynch mob of Teachatters heading your way
If you add spices to your tea, either Green or Black, then it becomes Masala Chai and then it IS a suitable vehicle for Milk and Sugar, if that enhances your drinking enjoyment.