We report on the discovery of HAT-P-11b, the smallest radius transiting extrasolar planet (TEP) discovered from the ground, and the first hot Neptune discovered to date by transit searches. HAT-P-11b orbits the bright (V=9.587) and metal rich ([Fe=H] = +0.31 +/- 0.05) K4 dwarf star GSC 03561-02092 with P = 4.8878162 +/- 0.0000071 days and produces a transit signal with depth of 4.2 mmag. We present a global analysis of the available photometric and radial-velocity data that result in stellar and planetary parameters, with simultaneous treatment of systematic variations. The planet, like its near-twin GJ 436b, is somewhat larger than Neptune (17Mearth, 3.8Rearth) both in mass Mp = 0.081 +/- 0.009 MJ (25.8 +/- 2.9 Mearth) and radius Rp = 0.422 +/- 0.014 RJ (4.73 +/- 0.16 Rearth). HAT-P-11b orbits in an eccentric orbit with e = 0.198 +/- 0.046 and omega = 355.2 +/- 17.3, causing a reflex motion of its parent star with amplitude 11.6 +/- 1.2 m/s, a challenging detection due to the high level of chromospheric activity of the parent star. Our ephemeris for the transit events is Tc = 2454605.89132 +/- 0.00032 (BJD), with duration 0.0957 +/- 0.0012 d, and secondary eclipse epoch of 2454608.96 +/- 0.15 d (BJD). The basic stellar parameters of the host star are M* = 0.809+0.020-0.027 Msun, R* = 0.752 +/- 0.021 Rsun and Teff = 4780 +/- 50 K. Importantly, HAT-P-11 will lie on one of the detectors of the forthcoming Kepler mission. We discuss an interesting constraint on the eccentricity of the system by the transit light curve and stellar parameters. We also present a blend analysis, that for the first time treats the case of a blended transiting hot Jupiter mimicing a transiting hot Neptune, and proves that HAT-P-11b is not such a blend.