The application of the Parallax of Pulsation (PoP) technique to determine the distances of pulsating stars implies the use of a scaling parameter, the projection factor (p-factor), that is required to transform disk-integrated radial velocities (RVs) into photospheric expansion velocities. The value of this parameter is poorly known and still debated. Most present applications of the PoP technique assume a constant p-factor. However, it may actually depend on the physical parameters of each star, as past studies aimed at calibrating the p-factor (usually for Cepheids) led to a broad range of individual values. We aim at calibrating the p-factors of a sample of RR Lyrae stars (RRLs), to compare them with classical Cepheids (CCs). Due to their higher surface gravity, RRLs have more compact atmospheres, thus providing a valuable comparison with their supergiant siblings. We determine the p-factor of 17 RR Lyrae stars, by modelling their pulsation using the SPIPS code. The models are constrained using Gaia DR3 parallaxes, photometry and new radial velocities that we collected with the OHP/SOPHIE spectrograph. We carefully examine the different steps of the PoP technique, in particular the method to determine the RV from spectra using the classical approach based on the cross- correlation function (CCF). The method employed to extract the radial velocity from the CCF has a strong impact on the p-factor, up to 10%. However, this choice of method results in a global scaling of the p-factor, and it affects only marginally the scatter of p within the sample for a given method. Over our RRL sample, we find a mean value of p=1.248+/-0.022 for RVs derived using a Gaussian fit of the CCF. There is no evidence for a different value of the p-factors of RRLs, although their distribution for RRLs appear significantly less scattered ({sigma}~7%) than those of CCs ({sigma}~12%). The p-factor does not appear to depend in a simple way on fundamental stellar parameters (pulsation period, radius, metallicity, amplitude of the radial velocity curve). We argue that large-amplitude dynamical phenomena occurring in the atmosphere of RRLs (and CCs) during their pulsation affect the relative velocity of the spectral line-forming regions compared to the velocity of the photosphere.