Divide the batter among the muffin cups. (Cupcakes will look about 1/2-full.) Spoon about a tablespoon of the filling into the center of each cupcake.

Vanilla frosting is pretty simple—some butter, some confectioners' sugar and a hint of vanilla—but when it comes to store-bought options, things get complicated quickly. Some out there are ...