We consider the free energy difference restricted to a finite volume for certain pairs of incongruent thermodynamic states (if they exist) in the Edwards-Anderson Ising spin glass at nonzero temperature. We prove that the variance of this quantity with respect to the couplings grows proportionally to the volume in any dimension greater than or equal to two. As an illustration of potential applications, we use this result to restrict the possible structure of Gibbs states in two dimensions.