We have systematically studied the magnetic properties of Cu$_{4-x}$Zn$_x$(OH)$_6$FBr by the neutron diffraction and muon spin rotation and relaxation ($mu$SR) techniques. Neutron-diffraction measurements suggest that the long-range magnetic order and the orthorhombic nuclear structure in the $x$ = 0 sample can persist up to $x$ = 0.23 and 0.43, respectively. The temperature dependence of the zero-field (ZF) $mu$SR spectra provide two characteristic temperatures, $T_{A0}$ and $T_{lambda}$. Comparison between $T_{A0}$ and $T_M$ from previously reported magnetic-susceptibility measurements suggest that the former comes from the short-range interlayer-spin clusters that persist up to $x$ = 0.82. On the other hand, the doping level where $T_{lambda}$ becomes zero is about 0.66, which is much higher than threshold of the long-range order, i.e., $sim$ 0.4. Our results suggest that the change in the nuclear structure may alter the spin dynamics of the kagome layers and a gapped quantum-spin-liquid state may exist above $x$ = 0.66 with the perfect kagome planes.