Lightly grease your baking dish with butter. Cut the bread into bite-sized cubes and spread them evenly in the dish. In a large bowl, whisk together eggs, heavy cream, milk, brown sugar, cinnamon, salt, and vanilla until well combined. Pour the custard evenly over the bread, making sure all pieces are coated. Cover tightly and refrigerate overnight to let the flavors soak in.
The next morning, mix the brown sugar, granulated sugar, cinnamon, and nutmeg in a small bowl. Add the cold butter pieces and use a pastry cutter or your hands to blend it into a crumbly texture. Keep the topping chilled until ready to use.
Preheat your oven to 350°F (180°C). Remove the casserole from the refrigerator and sprinkle the streusel topping evenly over the top. Bake uncovered for 45–50 minutes, or until the top is golden brown and the center is set. Let it cool for 10 minutes before serving so it holds its shape nicely.