CeRhIn$_5$ is a prototypical antiferromagnetic heavy-fermion compound, whose behavior in a magnetic field is unique. A magnetic field applied in the basal plane of the tetragonal crystal structure induces two additional phase transitions. When the magnetic field is applied along, or close to, the $c$ axis, a new phase characterized by a pronounced in-plane electronic anisotropy emerges at $B^* approx$ 30 T, well below the critical field, $B_c simeq$ 50 T, to suppress the antiferromagnetic order. The exact origin of this new phase, originally suggested to be an electronic-nematic state, remains elusive. Here we report low-temperature specific-heat measurements in CeRhIn$_5$ in high static magnetic fields up to 36 T applied along both the $a$ and $c$ axes. For fields applied along the $a$ axis, we confirmed the previously suggested phase diagram, and extended it to higher fields. This allowed us to observe a triple point at $sim$ 30 T, where the first-order transition from an incommensurate to commensurate magnetic structure merges into the onset of the second-order antiferromagnetic transition. For fields applied along the $c$ axis, we observed a small but distinct anomaly at $B^*$, which we discuss in terms of a possible field-induced transition, probably weakly first-order. We further suggest that the transition corresponds to a change of magnetic structure. We revise magnetic phase diagrams of CeRhIn$_5$ for both principal orientations of the magnetic field based entirely on thermodynamic anomalies.