The Radial Velocity Spectrometer (RVS) on board the ESA satellite mission Gaia has no calibration device. Therefore, the radial velocity zero point needs to be calibrated with stars that are proved to be stable at a level of 300m/s during the Gaia observations. We compiled a dataset of ~71000 radial velocity measurements from five high-resolution spectrographs. A catalogue of 4813 stars was built by combining these individual measurements. The zero point was established using asteroids. The resulting catalogue has seven observations per star on average on a typical time baseline of 6yr, with a median standard deviation of 15m/s. A subset of the most stable stars fulfilling the RVS requirements was used to establish the radial velocity zero point provided in Gaia Data Release 2. The stars that were not used for calibration are used to validate the RVS data.