The oxide heterostructure [(YFeO$_3$)$_5$(LaFeO$_3$)$_5$]$_{40}$, which is magnetically ordered and piezoelectric at room temperature, has been constructed from two weak ferromagnetic AFeO$_3$ perovskites with different A cations using RHEED-monitored pulsed laser deposition. The polarisation arises through the removal of inversion centres present within the individual AFeO$_3$ components. This symmetry reduction is a result of combining ordering on the A site, imposed by the periodicity of the grown structure, with appropriate orientations of the octahedral tilting characteristic of the perovskite units themselves, according to simple symmetry-controlled rules. The polarisation is robust against A site interdiffusion between the two layers which produces a sinusoidally modulated occupancy that retains the coupling of translational and point symmetries required to produce a polar structure. Magnetization and magneto-optical Kerr rotation measurements show that the heterostructures magnetic structure is similar to that of the individual components. Evidence of the polarity was obtained from second harmonic generation and piezoelectric force microscopy measurements. Modeling of the piezoresponse allows extraction of $d_{33}$ (approximately 10 pC/N) of the heterostructure, which is in agreement with DFT calculations.