Rare types of variable star may give unique insight into short-lived stages of stellar evolution. The systematic monitoring of millions of stars and advanced light curve analysis techniques of microlensing surveys make them ideal for discovering also such rare variable stars. One example is the R Coronae Borealis (RCB) stars, a rare type of evolved carbon-rich supergiant. We have conducted a systematic search of the EROS-2 database for the Galactic catalogue Bulge and spiral arms to find Galactic RCB stars. The light curves of $sim$100 million stars, monitored for 6.7 years (from July 1996 to February 2003), have been analysed to search for the main signature of RCB stars, large and rapid drops in luminosity. Follow-up spectroscopy has been used to confirm the photometric candidates. We have discovered 14 new RCB stars, all in the direction of the Galactic Bulge, bringing the total number of confirmed Galactic RCB stars to about 51. After reddening correction, the colours and absolute magnitudes of at least 9 of the stars are similar to those of Magellanic RCB stars. This suggests that these stars are in fact located in the Galactic Bulge, making them the first RCB stars discovered in the Bulge. The localisation of the 5 remaining RCBs is more uncertain: 4 are either located behind the Bulge at an estimated maximum distance of 14 kpc or have an unusual thick circumstellar shell; the other is a DY Per RCB which may be located in the Bulge, even if it is fainter than the known Magellanic DY Per. From the small scale height found using the 9 new Bulge RCBs, $61<h^{RCB}_{Bulge}<246$ pc (95% C.L.), we conclude that the RCB stars follow a disk-like distribution inside the Bulge.