The interactions between two octet baryons are studied at low energies using lattice QCD (LQCD) with larger-than-physical quark masses corresponding to a pion mass of $m_{pi}sim 450$ MeV and a kaon mass of $m_{K}sim 596$ MeV. The two-baryon systems that are analyzed range from strangeness $S=0$ to $S=-4$ and include the spin-singlet and triplet $NN$, $Sigma N$ ($I=3/2$), and $XiXi$ states, the spin-singlet $SigmaSigma$ ($I=2$) and $XiSigma$ ($I=3/2$) states, and the spin-triplet $Xi N$ ($I=0$) state. The $s$-wave scattering phase shifts, low-energy scattering parameters, and binding energies when applicable, are extracted using Luschers formalism. While the results are consistent with most of the systems being bound at this pion mass, the interactions in the spin-triplet $Sigma N$ and $XiXi$ channels are found to be repulsive and do not support bound states. Using results from previous studies at a larger pion mass, an extrapolation of the binding energies to the physical point is performed and is compared with experimental values and phenomenological predictions. The low-energy coefficients in pionless EFT relevant for two-baryon interactions, including those responsible for $SU(3)$ flavor-symmetry breaking, are constrained. The $SU(3)$ symmetry is observed to hold approximately at the chosen values of the quark masses, as well as the $SU(6)$ spin-flavor symmetry, predicted at large $N_c$. A remnant of an accidental $SU(16)$ symmetry found previously at a larger pion mass is further observed. The $SU(6)$-symmetric EFT constrained by these LQCD calculations is used to make predictions for two-baryon systems for which the low-energy scattering parameters could not be determined with LQCD directly in this study, and to constrain the coefficients of all leading $SU(3)$ flavor-symmetric interactions, demonstrating the predictive power of two-baryon EFTs matched to LQCD.