Heat the milk in a small saucepan until it just begins to simmer (don't boil). Remove from heat, add tea bags, and steep for 10-15 minutes. The milk should become noticeably fragrant and take on a light tan color.
Remove tea bags, squeezing gently to extract maximum flavor. Add butter to the warm tea-infused milk and stir until completely melted. Set aside to cool slightly.
Preheat your oven to 375°F (190°C). Line a 12-cup muffin tin with paper liners or generously grease each cup.
In a medium bowl, whisk together flour, baking powder, baking soda, salt, cinnamon, and nutmeg until well combined.
In a large bowl, whisk eggs, granulated sugar, and brown sugar until light and fluffy (about 2 minutes). Add the cooled tea-infused milk/butter mixture, oil, and vanilla extract. Whisk until smooth.
Gently fold in the dry ingredients until just combined – stop mixing as soon as no dry flour pockets remain.
Fold in the diced apples, ensuring they're evenly distributed throughout the batter.
If making the streusel topping: In a small bowl, combine flour, sugars, and cinnamon. Cut in cold butter using a pastry cutter or your fingertips until the mixture resembles coarse crumbs.
Divide batter evenly among prepared muffin cups, filling each about ¾ full. Sprinkle streusel topping over each muffin if using.
Bake for 20-22 minutes, or until a toothpick inserted into the center comes out clean with a few moist crumbs.
Allow muffins to cool in the pan for 5 minutes before transferring to a wire rack to cool completely.