(abridged) In this paper, we present the issues we consider as essential as far as the statistical mechanics of finite systems is concerned. In particular, we emphasis our present understanding of phase transitions in the framework of information theory. Information theory provides a thermodynamically-consistent treatment of finite, open, transient and expanding systems which are difficult problems in approaches using standard statistical ensembles. As an example, we analyze is the problem of boundary conditions, which in the framework of information theory must also be treated statistically. We recall that out of the thermodynamical limit the different ensembles are not equivalent and in particular they may lead to dramatically different equation of states, in the region of a first order phase transition. We recall the recent progresses achieved in the understanding of first-order phase transition in finite systems: the equivalence between the Yang-Lee theorem and the occurrence of bimodalities in the intensive ensemble and the presence of inverted curvatures of the thermodynamic potential of the associated extensive ensemble.