In Galactic open clusters, there is an apparent paucity of white dwarfs compared to the number expected assuming a reasonable initial mass function and that main-sequence stars with initial mass <= 8 M_sun become white dwarfs. We suggest that this lack of white dwarfs is due at least in part to dynamical processes. Non-spherically symmetric mass loss during the post-main-sequence evolution would lead to a few km/s isotropic recoil speed for the white dwarf remnant. This recoil speed can cause a substantial fraction of the white dwarfs formed in a cluster to leave the system. We investigate this dynamical process by carrying out high-precision N-body simulations of intermediate-mass open clusters, where we apply an isotropic recoil speed to the white dwarf remnants. Our models suggest that almost all white dwarfs would be lost from the cluster if the average recoil speed exceeds twice the velocity dispersion of the cluster.