Scaling laws of dust, HI gas, and metal mass with stellar mass, specific star formation rate, and metallicity are crucial to our understanding of the build-up of galaxies through their enrichment with metals and dust. In this work, we analyse how the dust and metal content varies with specific gas mass (M_HI_/M_*_) across a diverse sample of 423 nearby galaxies. The observed trends are interpreted with a set of Dust and Element evolUtion modelS (DEUS) - including stellar dust production, grain growth, and dust destruction - within a Bayesian framework to enable a rigorous search of the multidimensional parameter space. We find that these scaling laws for galaxies with -1.0~<logM_HI_/M_*_~<0 can be reproduced using closed-box models with high fractions (37-89 per cent) of supernova dust surviving a reverse shock, relatively low grain growth efficiencies ({epsilon}=30-40), and long dust lifetimes (1-2Gyr). The models have present-day dust masses with similar contributions from stellar sources (50-80 per cent) and grain growth (20-50 per cent). Over the entire lifetime of these galaxies, the contribution from stardust (>90 per cent) outweighs the fraction of dust grown in the interstellar medium (<10 per cent). Our results provide an alternative for the chemical evolution models that require extremely low supernova dust production efficiencies and short grain growth time-scales to reproduce local scaling laws, and could help solving the conundrum on whether or not grains can grow efficiently in the interstellar medium.