Using numerically exact methods we study transport in an interacting spin chain which for sufficiently strong spatially constant electric field is expected to experience Stark many-body localization. We show that starting from a generic initial state, a spin-excitation remains localized only up to a finite delocalization time, which depends exponentially on the size of the system and the strength of the electric field. This suggests that bona fide Stark many-body localization occurs only in the thermodynamic limit. We also demonstrate that the transient localization in a finite system and for electric fields stronger than the interaction strength can be well approximated by a Magnus expansion up-to times which grow with the electric field strength.