The mechanism by which galaxies stop forming stars and get rid of their interstellar medium (ISM) remains elusive. Here, we study a sample of more than 2000 elliptical galaxies in which dust emission has been detected. This is the largest sample of such galaxies ever analyzed. We infer the timescale for removal of dust in these galaxies and investigate its dependence on physical and environmental properties. We obtain a dust-removal timescale in elliptical galaxies of tau=2.26+/-0.18Gyr, corresponding to a half-life time of 1.57+/-0.12Gyr. This timescale does not depend on environment, stellar mass, or redshift. We observe a departure of dusty elliptical galaxies from the relation between star formation rate and dust mass. This is caused by the star formation rates declining faster than the dust masses and indicates that there exists an internal mechanism that affects star formation but leaves the ISM intact. Morphological quenching together with ionization or outflows caused by older stellar populations (Type Ia supernovae or planetary nebulae) is consistent with these observations.