The growth rate of the out-of-time-ordered correlator in a N-flavor Fermi gas is investigated and the Lyapunove exponent $lambda_L$ is calculated to the order of $1/N$. We find that the Lyapunove exponent monotonically increases as the the interaction strength increases from the BCS limit to the unitary region. At the unitarity the Lyapunove exponent increases while the temperature drops and it can reach to the order of $lambda_Lsim T$ around the critical temperature for the $N=1$ case. The system scrambles faster for stronger pairing fluctuations. At the BCS limit, the Lyapunov exponent behaviors as $lambda_Lpropto e^{mu/T} a^2_s T^2/N$.