We consider a local $U(1)_{B-L}$ extension of Zee-Babu model to explain the recently observed 3.5 keV X-ray line signal. The model has three Standard model (SM)-singlet Dirac fermions with different $U(1)_{B-L}$ charges. A complex scalar field charged under $U(1)_{B-L}$ is introduced to break the $U(1)_{B-L}$ symmetry. After $U(1)_{B-L}$ symmetry breaking a remnant discrete symmetry stabilizes the lightest state of the Dirac fermions, which can be a stable dark matter (DM). The second lightest state, if mass splitting with the stable DM is about 3.5 keV, decays dominantly to the stable DM and 3.5 keV photon through two-loop diagrams, explaining the X-ray line signal. Two-loop suppression of the decay amplitude makes its lifetime much longer than the age of the universe and it can be a decaying DM candidate in large parameter region. We also introduce a real scalar field which is singlet under both the SM and $U(1)_{B-L}$ and can explain the current relic abundance of the Dirac fermionic DMs. If the mixing with the SM Higgs boson is small, it does not contribute to DM direct detection. The main contribution to the scattering of DM off atomic nuclei comes from the exchange of $U(1)_{B-L}$ gauge boson, $Z$, and is suppressed below current experimental bound when $Z$ mass is heavy ($gtrsim 10$ TeV). If the singlet scalar mass is about 0.1-10 MeV, the DM self-interaction can be large enough to solve small scale structure problems in simulations with the cold DM, such as, the core-vs-cusp problem and too-big-to-fail problem.