We study the finite-temperature phase structure and the transition temperature of QCD with two flavors of dynamical quarks on a lattice with the temporal size $N_t=4$, using a renormalization group improved gauge action and the Wilson quark action improved by the clover term. The region of a parity-broken phase is identified, and the finite-temperature transition line is located on a two-dimensional parameter space of the coupling ($beta=6/g^2$) and hopping parameter $K$. Near the chiral transition point, defined as the crossing point of the critical line of the vanishing pion mass and the line of finite-temperature transition, the system exhibits behavior well described by the scaling exponents of the three-dimensional O(4) spin model. This indicates a second-order chiral transition in the continuum limit. The transition temperature in the chiral limit is estimated to be $T_c = 171(4)$ MeV.