We propose a hidden gauged $U(1)_H$ $Z$ model to explain deviations from the Standard Model (SM) values in lepton flavor universality known as $R_K$ and $R_D$ anomalies. The $Z$ only interacts with the SM fermions via their mixing with vector-like doublet fermions after the $U(1)_H$ symmetry breaking, which leads to $b to s mumu$ transition through the $Z^{prime}$ at tree level. Moreover, introducing an additional mediator, inert-Higgs doublet, yields $bto c tau u$ process via charged scalar contribution at tree level. Using flavio package, we scrutinize adequate sizes of the relevant Wilson coefficients to these two processes by taking various flavor observables into account. It is found that significant mixing between the vector-like and the second generation leptons is needed for the $R_K$ anomaly. A possible explanation of the $R_D$ anomaly can also be simultaneously addressed in a motivated situation, where a single scalar operator plays a dominant role, by the successful model parameters for the $R_K$ anomaly.