Preheat oven to 350° Fahrenheit. In a small bowl, stir together flour and baking powder. Set aside.
In the bowl of your mixer, blend 3/4 of the peanut butter with the room temperature butter on medium speed. Add the granulated sugar and brown sugar and beat till smooth, approx. 5 minutes. Beat in the egg until incorporated. Stir in the flour mixture until just combined.
Using a small ice cream scoop or a tablespoon, drop the dough 2 inches apart onto ungreased cookie sheets. Using your thumb, gently dent the center of the cookies. Spoon 1/2 tsp. of the strawberry jelly into each center. Bake until lightly golden, about 12 minutes. Remove to a cookie rack and cool.
In a small bowl, beat together the remaining 1/4 cup of peanut butter and the confectioners' sugar. With a fork or whisk, stir mixture while adding the milk, a little at a time, until the frosting reaches a good consistency for piping. Using a resealable bag with a tiny corner cut off, pipe the frosting over the cookies.