The first stars in the history of the Universe are likely to form in the dense central regions of {sim.to}10^5^-10^6^ M_{sun}_ cold dark matter halos at z{approx}10--50. The annihilation of dark matter particles in these environments may lead to the formation of so-called dark stars, which are predicted to be cooler, larger, more massive and potentially more long-lived than conventional population III stars. Here, we investigate the prospects of detecting high-redshift dark stars with the upcoming James Webb Space Telescope (JWST). We find that all dark stars with masses up to 10^3^ M_{sun}_ are intrinsically too faint to be detected by JWST at z above 6. However, by exploiting foreground galaxy clusters as gravitational telescopes, certain varieties of cool (T_eff_<=30000K) dark stars should be within reach at redshifts up to z{approx}10. If the lifetimes of dark stars are sufficiently long, many such objects may also congregate inside the first galaxies. We demonstrate that this could give rise to peculiar features in the integrated spectra of galaxies at high redshifts, provided that dark stars make up at least {sim.to}1% of the total stellar mass in such objects.