We derive a trade-off relation between the accuracy of implementing a desired unitary evolution using a restricted set of free unitaries and the size of the assisting system, in terms of the resource generating/losing capacity of the target unitary. In particular, this relation implies that, for any theory equipped with a resource measure satisfying lenient conditions, any resource changing unitary cannot be perfectly implemented by a free unitary applied to a system and an environment if the environment has finite dimensions. Our results are applicable to a wide class of resources including energy, asymmetry, coherence, entanglement, and magic, imposing ultimate limitations inherent in such important physical settings, as well as providing insights into operational restrictions in general resource theories.