Starting from the stochastic thermodynamics description of two coupled underdamped Brownian particles, we showcase and compare three different coarse-graining schemes leading to an effective thermodynamic description for the first of the two particles: marginalization over one particle, bipartite structure with information flows and the Hamiltonian of mean force formalism. In the limit of time-scale separation where the second particle with a fast relaxation time scale locally equilibrates with respect to the coordinates of the first slowly relaxing particle, the effective thermodynamics resulting from the first and third approach are shown to capture the full thermodynamics and to coincide with each other. In the bipartite approach, the slow part does not, in general, allow for an exact thermodynamic description as the entropic exchange between the particles is ignored. Physically, the second particle effectively becomes part of the heat reservoir. In the limit where the second particle becomes heavy and thus deterministic, the effective thermodynamics of the first two coarse-graining methods coincides with the full one. The Hamiltonian of mean force formalism however is shown to be incompatible with that limit. Physically, the second particle becomes a work source. These theoretical results are illustrated using an exactly solvable harmonic model.