Preheat oven to 350 degrees F. Line 18-20 cupcake cups with paper liners.
In a large mixing bowl, beat butter until creamy. Add peanut butter and both sugars and continue beating until light and fluffy. Beat in vanilla, then beat in eggs, one by one.
Mix together the flour, baking powder, baking soda and salt. Add to the batter alternately with the milk.
Divide batter evenly among the muffin/cupcake cups and bake on center rack for about 20 to 25 minutes or until cupcakes appear set.
Let the cupcakes cool, then use a sharp knife to carve out a little hole. Fill it with jelly.
Ice with Peanut Butter Cream Cheese Frosting. To make the frosting, beat together ½ cup peanut butter with ½ cup butter and 3 oz of softened cream cheese. Gradually stir in 2 cups of powdered sugar 1 teaspoon of vanilla extract and 2-3 tablespoons of milk Beat until light and fluffy. Taste to make sure it has enough peanut butter flavor, and if not, stir in some more.