We construct a $D_4$ flavor model based on SU(3)_C X SU}(3_L X U(1)_X gauge symmetry responsible for fermion masses and mixings. The neutrinos get small masses from antisextets which are in a singlet and a doublet under $D_4$. If the D_4 symmetry is violated as perturbation by a Higgs triplet under SU(3)_L and lying in {1} of D_4, the corresponding neutrino mass mixing matrix gets the most general form. In this case, the model can fit the experimental data in 2012 on neutrino masses and mixing. Our results show that the neutrino masses are naturally small and a little deviation from the tribimaximal neutrino mixing form can be realized. The quark masses and mixing matrix are also discussed. In the model under consideration, the CKM matrix can be different from the unit matrix. The scalar potential of the model is more simpler than those of the model based on $S_3$ and $S_4$. Assignation of VEVs to antisextets leads to the mixing of the new gauge bosons and those in the Standard Model. The mixing in the charged gauge bosons as well as the neutral gauge boson is considered.