We present electronic structure calculations together with resistivity, susceptibility, and specific heat measurements for TaB$_2$ to search for the recently contradictorily reported superconductivity and to study related normal state properties. We ascribe the absence of superconductivity down to 1.5 K for our TaB$_2$ samples to the generally weak electron phonon coupling derived from comparison of the calculated and measured specific heat constants. For the E$_{2g}$ and the B$_{1g}$ $Gamma$ point phonons we derive from the calculated deformation potentials very small electron phonon couplings for these modes, opposite to the strong coupling of the E$_{2g}$ mode in MgB$_2$, probably responsible for its high $T_c$. In comparison to MgB$_2$, we discuss the origin of the quite different features in the density of states and of the Fermi surfaces. The differences are mainly due to the strong hybridization between Ta 5$d$ and B 2$p$ states outside the hexagonal basis plane.