How does the host galaxy morphology influence a central quasar or vice versa? We address this question by measuring the asymmetries of 2424 SDSS quasar hosts at 0.2 < z < 0.8 using broad-band (grizy) images from the Hyper Suprime-Cam Subaru Strategic Program. Control galaxies (without quasars) are selected by matching the redshifts and stellar masses of the quasar hosts. A two-step pipeline is run to decompose the PSF and Sersic components and then measure asymmetry indices (ACAS, Aouter, and Ashape) of each quasar host and control galaxy. We find a mild correlation between host asymmetry and AGN bolometric luminosity (Lbol) for the full sample (spearman correlation of 0.37) while a stronger trend is evident at the highest luminosities (L_bol_>45). This then manifests itself into quasar hosts being more asymmetric, on average, when they harbour a more massive and highly accreting black hole. The merger fraction also positively correlates with Lbol and reaches up to 35 per cent for the most luminous. Compared to control galaxies, quasar hosts are marginally more asymmetric (excess of 0.017 in median at 9.4{sigma} level) and the merger fractions are similar (~16.5 per cent). We quantify the dependence of asymmetry on optical band that demonstrates that mergers are more likely to be identified with the bluer bands and the correlation between Lbol and asymmetry is also stronger in such bands. We stress that the band dependence, indicative of a changing stellar population, is an important factor in considering the influence of mergers on AGN activity.