The search for water-rich Earth-sized exoplanets around low-mass stars is rapidly gaining attention because they represent the best opportunity to characterize habitable planets in the near future. Understanding the atmospheres of these planets and determining the optimal strategy for characterizing them through transmission spectroscopy with our upcoming instrumentation is essential in order to constrain their environments. For this study, we present simulated transmission spectra of tidally locked Earth-sized ocean-covered planets around late-M to mid-K stellar spectral types, utilizing the results of general circulation models previously published by Kopparapu+ (2017ApJ...845....5K) as inputs for our radiative transfer calculations performed using NASA's Planetary Spectrum Generator (psg.gsfc.nasa.gov). We identify trends in the depth of H2O spectral features as a function of planet surface temperature and rotation rate. These trends allow us to calculate the exposure times necessary to detect water vapor in the atmospheres of aquaplanets through transmission spectroscopy with the upcoming James Webb Space Telescope (JWST) as well as several future flagship space telescope concepts under consideration (the Large UV Optical Infrared Surveyor (LUVOIR) and the Origins Space Telescope (OST)) for a target list constructed from the Transiting Exoplanet Survey Satellite (TESS) Input Catalog (TIC). Our calculations reveal that transmission spectra for water-rich Earth-sized planets around low-mass stars will be dominated by clouds, with spectral features <20ppm, and only a small subset of TIC stars would allow for the characterization of an ocean planet in the habitable zone. We thus present a careful prioritization of targets that are most amenable to follow-up characterizations with next-generation instrumentation, in order to assist the community in efficiently utilizing precious telescope time.