Based on first-principles calculations, we show that the maximum reachable concentration $x$ in the (Ga$_{1-x}$In$_x$)$_2$O$_3$ alloy in the low-$x$ regime (i.e. In solubility in $beta$-Ga$_2$O$_3$) is around 10%. We then calculate the band alignment at the (100) interface between $beta$-Ga$_2$O$_3$ and (Ga$_{1-x}$In$_x$)$_2$O$_3$ at 12%, the nearest computationally treatable concentration. The alignment is strongly strain-dependent: it is of type-B staggered when the alloy is epitaxial on Ga$_2$O$_3$, and type-A straddling in a free-standing superlattice. Our results suggest a limited range of applicability of low-In-content GaInO alloys.