Strain the custard through a fine sieve set over a heatproof bowl and let cool to room temperature. Whisk in the chilled buttermilk-peach mixture.