Check your ice cream maker manual for specific instructions. For my ice cream maker, you must freeze the freezer bowl for 24 hours before making ice cream.
In a small saucepan, heat the cream, milk, and vanilla extract on medium-low for 10 minutes. Be sure to stir occasionally to prevent boiling.
Meanwhile, in a large mixing bowl, add the yolks and sugar. With a whisk, break the yolks and combine with the sugar. Set aside.
Add the hot cocoa powder to the warmed milk mixture. Stir to combine. Let warm for 5-10 additional minutes or until the cocoa mixture is dissolved and combined. Strain the mixture into a heat-safe mixing bowl.
Working quickly, temper the yolk mixture with the hot cocoa mixture. To temper the yolks, whisk the eggs quickly while slowly adding the warm liquid (about 1/4 cup at a time) until all the liquid is combined.
Cover the bowl and place in the fridge. Let it chill for at least 2 hours and up to 24 hours.
Place the ice cream mixture into the ice cream maker and mix for 20 minutes, or until the ice cream is thickened to the desired consistency.
Serve immediately for soft serve, or place in a freezer safe container and freeze for a firmer texture. Top with marshmallows or your favorite topping before serving.