A neutron scattering study of nonsuperconducting La$_{2-x}$Sr$_x$CaCu$_2$O$_6$ (x=0 and 0.2), a bilayer copper oxide without CuO chains, has revealed an unexpected tetragonal-to-orthorhombic transition with a doping dependent transition temperature. The predominant structural modification below the transition is an in-plane shift of the apical oxygen. In the doped sample, the orthorhombic superstructure is strongly disordered, and a glassy state involving both magnetic and structural degrees of freedom develops at low temperature. The spin correlations are commensurate.