In a heavy saucepan, melt chocolate over low heat. Add milk; cook and stir until smooth. Stir in vanilla. Making one s'more at a time, spread 1 tablespoon chocolate mixture over each of two graham cracker halves.
Place eight or nine marshmallows on one cracker; gently press the other cracker on top. Repeat. Wrap in plastic; store at room temperature.