The origin of the fossil magnetic fields detected in 5 to 10% of intermediate-mass main sequence stars is still highly debated. We want to bring observational constraints to a large population of intermediate-mass pre-main sequence (PMS) stars in order to test the theory that convective-dynamo fields generated during the PMS phases of stellar evolution can occasionally relax into fossil fields on the main sequence. Using distance estimations, photometric measurements, and spectropolarimetric data from HARPSpol and ESPaDOnS of 38 intermediate-mass PMS stars, we determined fundamental stellar parameters (T_eff_, L and vsini) and measured surface magnetic field characteristics (including detection limits for non-detections, and longitudinal fields and basic topologies for positive detections). Using PMS evolutionary models, we determined the mass, radius, and internal structure of these stars. We compared different PMS models to check that our determinations were not model- dependant. We then compared the magnetic characteristics of our sample accounting for their stellar parameters and internal structures. Results: We detect magnetic fields in about half of our sample. About 90% of the magnetic stars have outer convective envelopes larger than ~25% of the stellar radii, and heavier than ~2% of the stellar mass. Going to higher mass, we find that the magnetic incidence in intermediate-mass stars drops very quickly, within a time-scale of the order of few times 0.1Myr. Finally, we propose that intermediate-mass T Tauri stars with large convective envelopes, close to the fully convective limit, have complex fields and that their dipole component strengths may decrease as the sizes of their convective envelopes decrease, similar to lower-mass T Tauri stars.