The elastic property of quantum critical quasicrystal (QC) Yb$_{15}$Al$_{34}$Au$_{51}$ is analyzed theoretically on the basis of the approximant crystal (AC) Yb$_{14}$Al$_{35}$Au$_{51}$. By constructing the realistic effective model in the AC, we evaluate the 4f-5d Coulomb repulsion at Yb as $U_{fd}approx 1.46$ eV realizing the quantum critical point (QCP) of the Yb-valence transition. The RPA analysis of the QCP shows that softening in elastic constants occurs remarkably for bulk modulus and longitudinal mode at low temperatures. Possible relevance of these results to the QC as well as the pressure-tuned AC is discussed.