In a saucepan, combine milk and yogurt. Add the pudding mix; cook and stir constantly until mixture boils and thickens. Stir in allspice. Add raisins; mix well and let cool for 10 minutes, stirring occasionally.
Pour into crust. Chill at least 2 hours. Serve with whipped topping if desired.