The structural and the magnetic properties of CeCu$_{6-x}$Ag$_x$ (0 $leq$ $x$ $leq$ 0.85) and CeCu$_{6-x}$Pd$_x$ (0 $leq$ $x$ $leq$ 0.4) have been studied using neutron diffraction, resonant ultrasound spectroscopy (RUS), heat capacity, x-ray diffraction measurements and first principles calculations. The structural and magnetic phase diagrams of CeCu$_{6-x}$Ag$_x$ and CeCu$_{6-x}$Pd$_x$ as a function of Ag/Pd composition are reported. The end member, CeCu$_6$, undergoes a structural phase transition from an orthorhombic ($Pnma$) to a monoclinic ($P2_1/c$) phase at 240 K. In CeCu$_{6-x}$Ag$_x$, the structural phase transition temperature (${T_{s}}$) decreases linearly with Ag concentration and extrapolates to zero at $x_{S}$ $approx$ 0.1. The structural transition in CeCu$_{6-x}$Pd$_x$ remains unperturbed with Pd substitution within the range of our study. The lattice constant $b$ slightly decreases with Ag/Pd doping, whereas, $a$ and $c$ increase with an overall increase in the unit cell volume. Both systems, CeCu$_{6-x}$Ag$_x$ and CeCu$_{6-x}$Pd$_x$, exhibit a magnetic quantum critical point (QCP), at $x$ $approx$ 0.2 and $x$ $approx$ 0.05 respectively. Near the QCP, long range antiferromagnetic ordering takes place at an incommensurate wave vector ($delta_1$ 0 $delta_2$) where $delta_1 sim 0.62$, $delta_2 sim 0.25$, $x$ = 0.125 for CeCu$_{6-x}$Pd$_x$ and $delta_1 sim 0.64$, $delta_2 sim 0.3$, $x$ = 0.3 for CeCu$_{6-x}$Ag$_x$. The magnetic structure consists of an amplitude modulation of the Ce-moments which are aligned along the $c$-axis of the orthorhombic unit cell.