A systematic investigation on muon anomalous magnetic moment and related lepton flavor-violating process such as $mto eg$, $tto eg$ and $tto mg$ is made at two loop level in the models with flavor-changing scalar interactions. The two loop diagrams with double scalar exchanges are studied and their contributions are found to be compatible with the ones from Barr-Zee diagram. By comparing with the latest data, the allowed ranges for the relevant Yukawa couplings $Y_{ij}$ in lepton sector are obtained. The results show a hierarchical structure of $Y_{m e, t e} ll Y_{m t} simeq Y_{mm}$ in the physical basis if $Delta a_{mu}$ is found to be $>50times 10^{-11}$. It deviates from the widely used ansatz in which the off diagonal elements are proportional to the square root of the products of related fermion masses. An alternative Yukawa coupling matrix in the lepton sector is suggested to understand the current data. With such a reasonable Yukawa coupling ansatz, the decay rate of $tto mg$ is found to be near the current experiment upper bound.