Based on the recently proposed band model, the electronic specific heat of moderately heavy electron compound YbAl$_3$ are investigated. The band term of the Hamiltonian consists of three parts; conduction electrons described by the nearly free electron method, localized 4f electrons of Yb ions and the hybridization term between these electrons. Extracting several bands near the Fermi level, we reconstruct the low-energy effective Hamiltonian in order to consider the correlation effect, which is studied by using the self-consistent second order perturbation theory combined with local approximation. The temperature dependence of the specific heat $c_{rm v}(T)$ is calculated as a function of temperature $T$ from the numerical derivative of the internal energy. Sommerfeld coefficient $gamma$ is also calculated from the direct formula. The overall structure of $c_{rm v}(T)/T$ is in quantitative agreement with the experimental results, which have the characteristic two-peak structures. They originate from the correlation effect and the structure of the non-interacting density of states, respectively. We show that our effective Hamiltonian yielding the realistic band structure may describe quantitatively heavy electron compounds with conduction bands composed of s- or p- electrons.