We compute the next-to-leading order (NLO) impact factor for inclusive photon $+$dijet production in electron-nucleus (e+A) deeply inelastic scattering (DIS) at small $x$. An important ingredient in our computation is the simple structure of ``shock wave fermion and gluon propagators. This allows one to employ standard momentum space Feynman diagram techniques for higher order computations in the Regge limit of fixed $Q^2gg Lambda_{rm QCD}^2$ and $xrightarrow 0$. Our computations in the Color Glass Condensate (CGC) effective field theory include the resummation of all-twist power corrections $Q_s^2/Q^2$, where $Q_s$ is the saturation scale in the nucleus. We discuss the structure of ultraviolet, collinear and soft divergences in the CGC, and extract the leading logs in $x$; the structure of the corresponding rapidity divergences gives a nontrivial first principles derivation of the JIMWLK renormalization group evolution equation for multiparton lightlike Wilson line correlators. Explicit expressions are given for the $x$-independent $O(alpha_s)$ contributions that constitute the NLO impact factor. These results, combined with extant results on NLO JIMWLK evolution, provide the ingredients to compute the inclusive photon $+$ dijet cross-section at small $x$ to $O(alpha_s^3 ln(x))$. First results for the NLO impact factor in inclusive dijet production are recovered in the soft photon limit. A byproduct of our computation is the LO photon+ 3 jet (quark-antiquark-gluon) cross-section.