We have synthesized four iron-based oxyarsenide superconductors Rb$Ln_2$Fe$_4$As$_4$O$_2$ ($Ln$ = Sm, Tb, Dy and Ho) resulting from the intergrowth of RbFe$_2$As$_2$ and $Ln$FeAsO. It is found that the lattice match between RbFe$_2$As$_2$ and $Ln$FeAsO is crucial for the phase formation. The structural intergrowth leads to double asymmetric Fe$_2$As$_2$ layers that are separated by insulating $Ln_2$O$_2$ slabs. Consequently, the materials are intrinsically doped at a level of 0.25 holes/Fe-atom and, bulk superconductivity emerges at $T_mathrm{c}$ = 35.8, 34.7, 34.3 and 33.8 K, respectively, for $Ln$ = Sm, Tb, Dy and Ho. Investigation on the correlation between crystal structure and $T_mathrm{c}$ suggests that interlayer couplings may play an additional role for optimization of superconductivity.