Cube the day-old bread into approximately 1-inch pieces. If using fresh bread, toast the cubes in a 350°F oven for about 10 minutes until slightly dried out. Transfer to a greased 9x13 inch baking dish.
In a large bowl, whisk together the eggs until frothy. Add the milk, heavy cream, maple syrup, vanilla extract, cinnamon, nutmeg, and salt. Whisk until completely uniform.
Peel, core, and dice the apples into 1/2-inch cubes. In a medium skillet over medium heat, melt the butter. Add the diced apples, brown sugar, cinnamon, and nutmeg. Cook for 5-7 minutes, stirring occasionally, until the apples are just tender but not mushy. Remove from heat and allow to cool slightly.
Pour about half of the custard mixture over the bread cubes, ensuring all pieces are moistened. Spoon the apple mixture evenly over the bread. Pour the remaining custard over the apples. Gently press down with a spatula to help the bread absorb the liquid.
In a medium bowl, combine flour, brown sugar, and cinnamon for the topping. Cut in the cold butter using a pastry cutter or two forks until the mixture resembles coarse crumbs. Stir in the chopped nuts if using.
Cover the casserole with plastic wrap and refrigerate for at least 2 hours or overnight.
When ready to bake, preheat your oven to 350°F. Remove the casserole from the refrigerator while the oven preheats. Sprinkle the crumb topping evenly over the soaked bread mixture.
Bake uncovered for 45-50 minutes, or until the center is set and the top is golden brown.
Let stand for 5-10 minutes before serving. Serve warm with maple syrup, fresh fruit, or a dollop of yogurt if desired.