In a small saucepan over medium heat, whisk together the cocoa powder, sugar, salt, and 2 tablespoons of the milk until smooth.
Gradually whisk in the rest of the milk and heat until hot but not boiling, stirring constantly.
Remove from heat and stir in the vanilla extract.
Pour into a mug and top with marshmallows or whipped cream if desired. Serve immediately.