Although optical metamaterials that show artificial magnetism are mesoscopic systems, they are frequently described in terms of effective material parameters. But due to intrinsic nonlocal (or spatially dispersive) effects it may be anticipated that this approach is usually only a crude approximation and is physically meaningless. In order to study the limitations regarding the assignment of effective material parameters, we present a technique to retrieve the frequency-dependent elements of the effective permittivity and permeability tensors for arbitrary angles of incidence and apply the method exemplarily to the fishnet metamaterial. It turns out that for the fishnet metamaterial, genuine effective material parameters can only be introduced if quite stringent constraints are imposed on the wavelength/unit cell size ratio. Unfortunately they are only met far away from the resonances that induce a magnetic response required for many envisioned applications of such a fishnet metamaterial. Our work clearly indicates that the mesoscopic nature and the related spatial dispersion of contemporary optical metamaterials that show artificial magnetism prohibits the meaningful introduction of conventional effective material parameters.