Vector beams (VBs) are widely investigated for its special intensity and polarization distributions, which is useful for optical micromanipulation, optical micro-fabrication, optical communication, and single molecule imaging. To date, it is still a challenge to realize nonlinear frequency conversion (NFC) and manipulation of such VBs because of the polarization sensitivity in most of nonlinear processes. Here, we report an experimental realization of NFC and manipulation of VBs which can be used to expand the available frequency band. The main idea of our scheme is to introduce a Sagnac loop to solve the polarization dependence of NFC in nonlinear crystals. Furthermore, we find that a linearly polarized vector beam should be transformed to an exponential form before performing the NFC. The experimental results are well agree with our theoretical model. The present method is also applicable to other wave bands and second order nonlinear processes, and may also be generalized to the quantum regime for single photons.