We report our recent lattice calculation of hadronic light-by-light contribution to muon $g-2$ using our recently developed moment method. The connected diagrams and the leading disconnected diagrams are included. The calculation is performed on a $48^3 times 96$ lattice with physical pion mass and 5.5 fm box size. We expect sizable finite volume and finite lattice spacing corrections to the results of these calculations which will be estimated in calculations to be carried out over the next 1-2 years.