I study the universal finite-size scaling function for the lowest gap of the quantum Ising chain with a one-parameter family of ``defect boundary conditions, which includes periodic, open, and antiperiodic boundary conditions as special cases. The universal behavior can be described by the Majorana fermion field theory in $1+1$ dimensions, with the mass proportional to the deviation from the critical point. Although the field theory appears to be symmetric with respect to the inversion of the mass (Kramers-Wannier duality), the actual gap is asymmetric, reflecting the spontaneous symmetry breaking in the ordered phase which leads to the two-fold ground-state degeneracy in the thermodynamic limit. The asymptotic ground-state degeneracy in the ordered phase is realized by (i) formation of a bound state at the defect (except for the periodic/antiperiodic boundary condition) and (ii) effective reversal of the fermion number parity in one of the sectors (except for the open boundary condition), resulting in a rather nontrivial crossover ``phase diagram in the space of the boundary condition (defect strength) and mass.