Strong gravitational lensing provides an independent measurement of the Hubble parameter (H_0_). One remaining systematic is a bias from the additional mass due to a galaxy group at the lens redshift or along the sightline. We quantify this bias for more than 20 strong lenses that have well-sampled sightline mass distributions, focusing on the convergence {kappa} and shear {gamma}. In 23% of these fields, a lens group contributes >=1% convergence bias; in 57%, there is a similarly significant line-of-sight group. For the nine time-delay lens systems, H0 is overestimated by 11_-2_^+3^% on average when groups are ignored. In 67% of fields with total {kappa}>=0.01, line-of-sight groups contribute >~2x more convergence than do lens groups, indicating that the lens group is not the only important mass. Lens environment affects the ratio of four (quad) to two (double) image systems; all seven quads have lens groups while only 3 of 10 doubles do, and the highest convergences due to lens groups are in quads. We calibrate the {gamma}-{kappa} relation: log({kappa}_tot_)=(1.94+/-0.34)log({gamma}_tot_)+(1.31+/-0.49) with an rms scatter of 0.34dex. Although shear can be measured directly from lensed images, unlike convergence, it can be a poor predictor of convergence; for 19% of our fields, {kappa} is >~2{gamma}. Thus, accurate cosmology using strong gravitational lenses requires precise measurement and correction for all significant structures in each lens field.