Preheat oven to 375°F (190°C). Grease or line a muffin tin with paper liners.
Separate crescent roll dough into 8 triangles.
In a small bowl, mix sugar and cinnamon together.
Dip each marshmallow in melted butter, then roll in cinnamon sugar.
Place each marshmallow on the wide end of a crescent triangle and roll up, pinching all seams to seal completely.
Place seam-side down in the muffin tin.
Bake for 10–12 minutes or until golden brown.
Let cool slightly, then mix powdered sugar, milk, and vanilla to make a glaze.
Drizzle glaze over warm puffs and serve.