Surface states generally degrade semiconductor device performance by raising the charge injection barrier height, introducing localized trap states, inducing surface leakage current, and altering the electric potential. Therefore, there has been an endless effort to use various surface passivation treatments to suppress the undesirable impacts of the surface states. We show that the giant built-in electric field created by the surface states can be harnessed to enable passive wavelength conversion without utilizing any nonlinear optical phenomena. Photo-excited surface plasmons are coupled to the surface states to generate an electron gas, which is routed to a nanoantenna array through the giant electric field created by the surface states. The induced current on the nanoantennas, which contains mixing product of different optical frequency components, generates radiation at the beat frequencies of the incident photons. We utilize the unprecedented functionalities of plasmon-coupled surface states to demonstrate passive wavelength conversion of nanojoule optical pulses at a 1550 nm center wavelength to terahertz regime with record-high efficiencies that exceed nonlinear optical methods by 4-orders of magnitude. The presented scheme can be used for optical wavelength conversion to different parts of the electromagnetic spectrum ranging from microwave to infrared regimes by using appropriate optical beat frequencies.