Isolated photons with high transverse energy have been studied in deep inelastic $ep$ scattering with the ZEUS detector at HERA, using an integrated luminosity of $326,$ pb$^{-1}$ in the range of exchanged-photon virtuality $10 - 350$ GeV$^2$. Outgoing isolated photons with transverse energy $4<E_T^gamma< 15$ GeV and pseudorapidity $-0.7 <eta^gamma< 0.9$ were measured with accompanying jets having transverse energy and pseudorapidity $2.5 <E_T^{jet}<35$ GeV and $-1.5<eta^{jet}< 1.8$, respectively. Differential cross sections are presented for the following variables: the fraction of the incoming photon energy and momentum that is transferred to the outgoing photon and the leading jet; the fraction of the incoming proton energy transferred to the photon and leading jet; the differences in azimuthal angle and pseudorapidity between the outgoing photon and the leading jet and between the outgoing photon and the scattered electron. Comparisons are made with theoretical predictions: a leading-logarithm Monte Carlo simulation, a next-to-leading-order QCD prediction, and a prediction using the $k_T$-factorisation approach.