We present a weak lensing analysis for galaxy clusters from the APEX-SZ survey. For 39 massive galaxy clusters that were observed via the Sunyaev-Zel'dovich effect (SZE) with the APEX telescope, we analyse deep optical imaging data from WFI(@2.2mMPG/ESO) and Suprime-Cam(@SUBARU) in three bands. The masses obtained in this study, including an X-ray selected sub-sample of 27 clusters, are optimized for and used in studies constraining the mass to observable scaling relations at fixed cosmology. A novel focus of our weak lensing analysis is the multicolour background selection to suppress effects of cosmic variance on the redshift distribution of source galaxies. We investigate the effects of cluster member contamination through galaxy density, shear profile, and recovered concentrations. We quantify the impact of variance in source redshift distribution on the mass estimate by studying nine sub-fields of the COSMOS survey for different cluster redshift and magnitude limits. We measure a standard deviation of ~6 per cent on the mean angular diameter distance ratio for a cluster at z=0.45 and shallow imaging data of R~23mag. It falls to ~1 per cent for deep, R=26mag, observations. This corresponds to 8.4 per cent and 1.4 per cent scatter in M_200_. Our background selection reduces this scatter by 20-40 per cent, depending on cluster redshift and imaging depth. We derived cluster masses with and without using a mass concentration relation and find consistent results, and concentrations consistent with the used mass-concentration relation.