Athermal systems across a large range of length scales, ranging from foams and granular bead packings to crumpled metallic sheets, exhibit slow stress relaxation when compressed. Experimentally they show a non-monotonic stress response when decompressed somewhat after an initial compression, i.e. under a two-step, Kovacs-like protocol. It turns out that from this response one can tell the age of the system, suggesting an interpretation as a memory effect. In this work we use a model of an athermal jammed solid, specifically a binary mixture of soft harmonic spheres, to explore this phenomenon through in-silico experiments. Using extensive simulations under conditions analogous to those in experiment, we observe identical phenomenology in the stress response under a two--step protocol. Our model system also recovers the behaviour under a more recently studied three-step protocol, which consists of a compression followed by a decompression and then a final compression. We show that the observed response in both two-step and three-step protocols can be understood using Linear Response Theory. In particular, a linear scaling with age for the two-step protocol arises generically for slow linear responses with power law or logarithmic decay and does not in itself point to any underlying aging dynamics.