Mix yolks, sugar, cornstarch, and salt. In a medium saucepan, whisk together the egg yolks, light brown sugar, cornstarch, and salt until smooth and well combined.
Slowly add milk. Gradually whisk in the whole milk, a little at a time, to keep the mixture smooth and lump-free.
Cook, whisking constantly, until thick. Place the saucepan over medium heat and cook, whisking constantly, until the mixture thickens and begins to bubble, about 8 to 10 minutes.
Cook 1-2 more minutes. Once the pudding has thickened, reduce the heat to low and continue cooking for 1 to 2 minutes, whisking constantly, to cook out the raw cornstarch flavor.
Add butter and vanilla. Remove the saucepan from the heat and whisk in the cubed butter and vanilla extract until fully incorporated and smooth.
Strain. Strain the pudding through a fine-mesh sieve into a large bowl to remove any lumps and ensure a silky texture.
Divide and cover. Divide the pudding evenly among 6 serving dishes or ramekins. Cover each dish with plastic wrap, pressing the wrap directly onto the surface of the pudding to prevent a skin from forming.
Chill and serve. Refrigerate for at least 2 hours, or until fully chilled and set. Serve cold, optionally topped with whipped cream or a sprinkle of flaky sea salt.