We present a calculation of the kaon form factors in two-flavor QCD with the non-perturbatively $O(a)$-improved Wilson quark action. In order to achieve a few percent accuracy in the study of SU(3) breaking effects, we use a set of double ratios of the matrix elements, with which the bulk of the statistical fluctuation and the multiplicative renormalization factors cancel.