The dust-to-stellar mass ratio (M_dust_/M*) is a crucial yet poorly constrained quantity to understand the complex physical processes involved in the production of dust, metals and stars in galaxy evolution. In this work we explore trends of M_dust_/M* with different physica parameters using observations of 300 massive, dusty star-forming galaxies detected with ALMA up to z~5. Additionally, we interpret our findings with different models of dusty galaxy formation. We find that M_dust_/M* evolves with redshift, stellar mass, specific star formation rate and integrated dust size, differently for main sequence and starburst galaxies. In both galaxy populations M_dust_/M* increases until z~2 followed by a roughly flat trend towards higher redshifts, suggesting efficient dust growth in the distant universe. We confirm that the inverse relation between M_dust_/M* and M* holds up to z~5 and can be interpreted as an evolutionary transition from early to late starburst phases. We demonstrate that M_dust_/M* in starbursts reflects the increase in molecular gas fraction with redshift, and attains the highest values for sources with the most compact dusty star-formation. The state-of-the-art cosmological simulations that include self-consistent dust growth, broadly reproduce the evolution of M_dust_/M* in main sequence galaxies, but underestimate it in starbursts. The latter is found to be linked to lower gas-phase metallicities and longer dust growth timescales relative to observations. Phenomenological models based on the main-sequence/starburst dichotomy and analytical models that include recipes for rapid metal enrichment are consistent with our observations. Therefore, our results strongly suggest that high M_dust_/M* is due to rapid dust grain growth in metal enriched interstellar medium. This work highlights multifold benefits of using M_dust_/M* as a diagnostic tool for: (1) disentangling main sequence and starburst galaxies up to z~5; (2) probing the evolutionary phase of massive objects; and (3) refining the treatment of the dust life cycle in simulations.