Pulsars typically exhibit radio emission in the form of narrow pulses originated from confined regions of their magnetospheres. A potential presence of magnetospherically originated emission outside this region, the so-called off-pulse emission, would challenge the existing theories. Detection of significant off-pulse emission has been reported so far from only two pulsars, B0525+21 and B2045-16, at 325 and 610 MHz. However, the nature of this newly uncovered off-pulse emission remains unclear. To probe its origin we conducted very high resolution radio observations of B0525+21 and B2045-16 with the European VLBI Network (EVN) at 1.39 GHz. Whereas the pulsed emission is detected at a level consistent with previous observations, we report absence of any off-pulse emission above $42$ and $96 mathrm{mu Jy beam^{-1}}$ (three times the rms noise levels) for B0525+21 and B2045-16, respectively. Our stringent upper limits imply the off-pulse emission to be less than $0.4$ and $0.3%$ of the period-averaged pulsed flux density, i.e., much fainter than the previously suggested values of $1$-$10%$. Since the EVN data are most sensitive to extremely compact angular scales, our results suggest a non-magnetospheric origin for the previously reported off-pulse emission. Presence of extended emission that is resolved out on these milliarcsecond scales still remains plausible. In this case, we constrain the emission to arise from structures with sizes of $sim (0.61$-$19) times 10^3 mathrm{au}$ for B0525+21 and $sim (0.48$-$8.3) times 10^3 mathrm{au}$ for B2045-16. These constraints might indicate that the two pulsars are accompanied by compact bow-shock pulsar wind nebulae. Future observations probing intermediate angular scales ($sim 0.1$-$5 mathrm{arcsec}$) will help in clarifying the actual origin of the off-pulse emission.