Context: Asteroseismology has entered a new era with the advent of the NASA Kepler mission. Long and continuous photometric observations of unprecedented quality are now available which have stimulated the development of a number of suites of innovative analysis tools. Aims: The power spectra of solar-like oscillations are an inexhaustible source of information on stellar structure and evolution. Robust methods are hence needed in order to infer both individual oscillation mode parameters and parameters describing non-resonant features, thus making a seismic interpretation possible. Methods: We present a comprehensive guide to the implementation of a Bayesian peak-bagging tool that employs a Markov chain Monte Carlo (MCMC). Besides making it possible to incorporate relevant prior information through Bayes theorem, this tool also allows one to obtain the marginal probability density function for each of the fitted parameters. We apply this tool to a couple of recent asteroseismic data sets, namely, to CoRoT observations of HD 49933 and to ground-based observations made during a campaign devoted to Procyon. Results: The developed method performs remarkably well at constraining not only in the traditional case of extracting oscillation frequencies, but also when pushing the limit where traditional methods have difficulties. Moreover it provides an rigorous way of comparing competing models, such as the ridge identifications, against the asteroseismic data.