Preheat the oven to 350°F and grease your baking dish well. In one bowl, sift together the flour, baking powder, and salt to remove lumps. Separating dry ingredients ensures the sponge bakes evenly and stays light.
Beat the egg yolks with sugar until pale and creamy, then mix in the milk, ube extract, and ube halaya. In another bowl, whip egg whites with sugar until stiff peaks form. This step is key to achieving a fluffy cake.
Gently fold the yolk mixture into the dry ingredients, then carefully fold in the whipped egg whites. Pour the batter into the baking dish and bake until a toothpick inserted in the center comes out clean. Let the cake cool slightly.
Mix evaporated milk, condensed milk, coconut milk, and ube extract in a bowl. Poke holes all over the cake and slowly pour the milk mixture on top. Allow the cake to absorb the liquid, then refrigerate for at least eight hours.
Whip heavy cream, coconut milk, sugar, and ube extract until soft to medium peaks form. Spread evenly over the chilled cake and smooth the surface before serving.