We present the first continuum extrapolation of the hyperon octet axial couplings ($g_{Sigma Sigma}$ and $g_{Xi Xi}$) from $N_f=2+1+1$ lattice QCD. These couplings are important parameters in the low-energy effective field theory description of the octet baryons and fundamental to the nonleptonic decays of hyperons and to hyperon-hyperon and hyperon-nucleon scattering with application to neutron stars. We use clover lattice fermion action for the valence quarks with sea quarks coming from configurations of $N_f=2+1+1$ highly improved staggered quarks (HISQ) generated by MILC Collaboration. Our work includes the first calculation of $g_{Sigma Sigma}$ and $g_{Xi Xi}$ directly at the physical pion mass on the lattice, and a full account of systematic uncertainty, including excited-state contamination, finite-volume effects and continuum extrapolation, all addressed for the first time. We find the continuum-limit hyperon coupling constants to be $g_{Sigma Sigma}=0.4455(55)_text{stat}(65)_text{sys}$ and $g_{Xi Xi} =-0.2703(47)_text{stat}(13)_text{sys}$, which correspond to low-energy constants of $D = 0.708(10)_text{stat}(6)_text{sys}$ and $F = 0.438(7)_text{stat}(6)_text{sys}$. The corresponding SU(3) symmetry breaking is 9% which is about a factor of 2 smaller than the earlier lattice estimate.