A learning-based posterior distribution estimation method, Probabilistic Dipole Inversion (PDI), is proposed to solve quantitative susceptibility mapping (QSM) inverse problem in MRI with uncertainty estimation. A deep convolutional neural network (CNN) is used to represent the multivariate Gaussian distribution as the approximated posterior distribution of susceptibility given the input measured field. In PDI, such CNN is firstly trained on healthy subjects dataset with labels by maximizing the posterior Gaussian distribution loss function as used in Bayesian deep learning. When tested on new dataset without any label, PDI updates the pre-trained network in an unsupervised fashion by minimizing the KL divergence between the approximated posterior distribution represented by CNN and the true posterior distribution given the likelihood distribution from known physical model and prior distribution. Based on our experiments, PDI provides additional uncertainty estimation compared to the conventional MAP approach, meanwhile addressing the potential discrepancy issue of CNN when test data deviates from training dataset.