In a small bowl, combine cracker crumbs and sugar; stir in butter. Set aside 1/2 cup for topping. Press remaining crumb mixture onto the bottom and up the sides of a greased 9-in. springform pan or 9-in. square baking pan. Bake at 350° for 5-7 minutes. Cool on wire rack.
In a large bowl, beat cream cheese and sugar until smooth. Fold in 2 cups whipped topping. Arrange half of the banana slices in crust; top with half of the cream cheese mixture. Repeat layers.
In a small bowl, whisk milk and pudding mix for 2 minutes. Let stand for 2 minutes or until soft-set; fold in remaining whipped topping. Pour over the cream cheese layer. Sprinkle with reserved crumb mixture. Refrigerate for 1-2 hours or until set.