Stabilized cat codes can provide a biased noise channel with a set of bias-preserving (BP) gates, which can significantly reduce the resource overhead for fault-tolerant quantum computing. All existing schemes of BP gates, however, require adiabatic quantum evolution, with performance limited by excitation loss and non-adiabatic errors during the adiabatic gates. In this work, we apply a derivative-based leakage suppression technique to overcome non-adiabatic errors, so that we can implement fast BP gates on Kerr-cat qubits with improved gate fidelity while maintaining high noise bias. When applied to concatenated quantum error correction, the fast BP gates can not only improve the logical error rate but also reduce resource overhead, which enables more efficient implementation of fault-tolerant quantum computing.