Preheat oven to 350°F (175°C). Grease a 9x5-inch loaf pan or line with parchment paper.
In a medium bowl, whisk together flour, pumpkin pie spice, baking soda, baking powder, and salt.
In a large bowl, combine pumpkin puree, granulated sugar, brown sugar, eggs, oil, milk, and vanilla extract until smooth.
Gradually mix dry ingredients into the wet ingredients until just combined. Do not overmix.
Pour batter into the prepared loaf pan and sprinkle with nuts or pepitas, if using.
Bake for 60–65 minutes or until a toothpick inserted in the center comes out clean.
Allow to cool in the pan for 15 minutes, then transfer to a wire rack to cool completely.