We study the non-equilibrium thermodynamics of a heat engine operating between two finite-sized reservoirs with well-defined temperatures. Within the linear response regime, it is discovered that there exists a power-efficiency trade-off depending on the ratio of heat capacities ($gamma$) of the reservoirs for the engine; the uniform temperature of the two reservoirs at final time $tau$ is bounded from below by the entropy production $sigma_{mathrm{min}}propto1/tau$. We further obtain a universal efficiency at maximum power of the engine for arbitrary $gamma$. Our findings can be used to develop an optimization scenario for thermodynamic cycles with finite-sized reservoirs in practice.