When modeling laser wakefield acceleration (LWFA) using the particle-in-cell (PIC) algorithm in a Lorentz boosted frame, the plasma is drifting relativistically at $beta_b c$ towards the laser, which can lead to a computational speedup of $sim gamma_b^2=(1-beta_b^2)^{-1}$. Meanwhile, when LWFA is modeled in the quasi-3D geometry in which the electromagnetic fields and current are decomposed into a limited number of azimuthal harmonics, speedups are achieved by modeling three dimensional problems with the computation load on the order of two dimensional $r-z$ simulations. Here, we describe how to combine the speed ups from the Lorentz boosted frame and quasi-3D algorithms. The key to the combination is the use of a hybrid Yee-FFT solver in the quasi-3D geometry that can be used to effectively eliminate the Numerical Cerenkov Instability (NCI) that inevitably arises in a Lorentz boosted frame due to the unphysical coupling of Langmuir modes and EM modes of the relativistically drifting plasma in these simulations. In addition, based on the space-time distribution of the LWFA data in the lab and boosted frame, we propose to use a moving window to follow the drifting plasma to further reduce the computational load. We describe the details of how the NCI is eliminated for the quasi-3D geometry, the setups for simulations which combine the Lorentz boosted frame and quasi-3D geometry, the use of a moving window, and compare the results from these simulations against their corresponding lab frame cases. Good agreement is obtained, particularly when there is no self-trapping, which demonstrates it is possible to combine the Lorentz boosted frame and the quasi-3D algorithms when modeling LWFA to achieve unprecedented speedups.