The anharmonicity of phonons in solid is ultimately rooted in the chemical bonding. However, the direct connection between phonon anharmoncity and chemical bonding is difficult to make experimentally or theoretically, due mainly to their complicated lattice structures. Here, with the help of density functional theory based calculations, we discovery that electrostatic inter-layer coupling in Bi$_2$O$_2$X (X=S,Se,Te) leads to intrinsically low lattice thermal conductivity. We explain our discovery by the strong anharmonic chemical bonding between Bi and chalcogen atoms. Our results shed light on the connection between inter-layer chemical bonding and phonon anharmonicity, which could be explored in a wide range of layered materials.