It is well known that in Anderson localized systems, starting from a random product state the entanglement entropy remains bounded at all times. However, we show that adding a single boundary term to an otherwise Anderson localized Hamiltonian leads to unbounded growth of entanglement. Our results imply that Anderson localization is not a local property. One cannot conclude that a subsystem has Anderson localized behavior without looking at the whole system, as a term that is arbitrarily far from the subsystem can affect the dynamics of the subsystem in such a way that the features of Anderson localization are lost.