Intermediate-mass stars (IMSs) represent the link between low-mass and high-mass stars, and cover a key mass range for giant planet formation. In this paper, we present a spectroscopic survey of 241 young IMS candidates with IR-excess, the most complete unbiased sample to date within 300 pc. We combined VLT/X-Shooter spectra with BVR photometric observations and Gaia DR3 distances to estimate fundamental stellar parameters such as Teff, mass, radius, age, and luminosity. We further selected those stars within the intermediate-mass range 1.5-3.5 solar masses, and discarded old contaminants. We used 2MASS and WISE photometry to study the IR-excesses of the sample, finding 92 previously unidentified stars with IR-excess. We classified this sample into "protoplanetary", "hybrid candidates", and "debris" discs based on their observed fractional excess at 12 microns, finding a new population of 17 hybrid disc candidates. We studied inner disc dispersal time-scales for wavelengths under 10 microns and found very different trends for IMSs and low-mass stars (LMSs). IMSs show excesses dropping fast during the first 6Myr independently of the wavelength, while LMSs show consistently lower fractions of excess at the shortest wavelengths, and increasingly higher fractions for longer wavelengths with slower dispersal rates. In conclusion, this study demonstrates empirically that IMSs dissipate their inner discs very differently than LMSs, providing a possible explanation for the lack of short period planets around IMSs.