Verifying that sub-mm galaxies are gravitationally lensed requires time-expensive observations with oversubscribed high-resolution observatories. Here, we aim to strengthen the evidence of gravitational lensing within the Herschel Bright Sources (HerBS) by cross-comparing their positions to optical (SDSS) and near-infrared (VIKING) surveys, in order to search for the foreground lensing galaxy candidates. Resolved observations of the brightest HerBS sources have already shown that most are lensed, and a galaxy evolution model predicts that ~76 per cent of the total HerBS sources are lensed, although with the SDSS survey we are only able to identify the likely foreground lenses for 25 per cent of the sources. With the near-infrared VIKING survey, however, we are able to identify the likely foreground lenses for 57 per cent of the sources, and we estimate that 82 per cent of the HerBS sources have lenses on the VIKING images even if we cannot identify the lens in every case. We find that the angular offsets between lens and Herschel source are larger than that expected if the lensing is done by individual galaxies. We also find that the fraction of HerBS sources that are lensed falls with decreasing 500-micron flux density, which is expected from the galaxy evolution model. Finally, we apply our statistical VIKING cross-identification to the entire Herschel-ATLAS catalogue, where we also find that the number of lensed sources falls with decreasing 500-micron flux density.