The study of the planet-debris disk connection can shed light on the formation and evolution of planetary systems and may help "predict" the presence of planets around stars with certain disk characteristics. In preliminary analyses of subsamples of the Herschel DEBRIS and DUNES surveys, Wyatt et al. (2012MNRAS.424.1206W) and Marshall et al. (2014A&A...565A..15M) identified a tentative correlation between debris and the presence of low-mass planets. Here we use the cleanest possible sample out of these Herschel surveys to assess the presence of such a correlation, discarding stars without known ages, with ages <1Gyr, and with binary companions <100AU to rule out possible correlations due to effects other than planet presence. In our resulting subsample of 204 FGK stars, we do not find evidence that debris disks are more common or more dusty around stars harboring high-mass or low-mass planets compared to a control sample without identified planets.