We report a detailed analysis of the orbital properties of binary millisecond pulsar (MSP) with a white dwarf (WD) companion. Positive correlations between the orbital period $P_{rm b}$ and eccentricity $epsilon$ are found in two classes of MSP binaries with a He WD and with a CO/ONeMg WD, though their trends are different. The distribution of $P_{rm b}$ is not uniform. Deficiency of sources at $P_{rm b}sim35-50$~days (Gap 1) have been mentioned in previous studies. On the other hand, another gap at $P_{rm b}sim2.5-4.5$~days (Gap 2) is identified for the first time. Inspection of the relation between $P_{rm b}$ and the companion masses $M_{rm c}$ revealed the subpopulations of MSP binaries with a He WD separated by Gap 1, above which $P_{rm b}$ is independent of $M_{rm c}$ (horizontal branch) but below which $P_{rm b}$ correlates strongly with $M_{rm c}$ (lower branch). Distinctive horizontal branch and lower branch separated by Gap 2 were identified for the MSP binaries with a CO/ONeMg WD at shorter $P_{rm b}$ and higher $M_{rm c}$. Generally, $M_{rm c}$ are higher in the horizontal branch than in the lower branch for the MSP binaries with a He WD. These properties can be explained in terms of a binary orbital evolution scenario in which the WD companion was ablated by a pulsar wind in the post mass-transfer phase.