We estimate the shear and the bulk viscous coefficients for a hot hadronic gas mixture constituting of pions and nucleons. The viscosities are evaluated in the relativistic kinetic theory approach by solving the transport equation in the relaxation time approximation for binary collisions ($pipi$,$pi N$ and $NN$). Instead of vacuum cross-sections usually used in the literature we employ in-medium scattering amplitudes in the estimation of the relaxation times. The modified cross-sections for $pipi$ and $pi N$ scattering are obtained using one-loop modified thermal propagators for $rho$, $sigma$ and $Delta$ in the scattering amplitudes which are calculated using effective interactions. The resulting suppression of the cross sections at finite temperature and baryon density is observed to significantly affect the $T$ and $mu_N$ dependence of the viscosities of the system.