We present an atlas of Spitzer/IRS high-resolution (R~600) 10-37um spectra for 24 well known starburst galaxies. The spectra are dominated by fine-structure lines, molecular hydrogen lines, and emission bands of polycyclic aromatic hydrocarbons (PAHs). Six out of the eight objects with a known active galactic nucleus (AGN) component show emission of the high excitation [NeV] line. This line is also seen in one other object (NGC 4194) with, a priori, no known AGN component. In addition to strong PAH emission features in this wavelength range (11.3, 12.7, 16.4um), the spectra reveal other weak hydrocarbon features at 10.6, 13.5, 14.2um, and a previously unreported emission feature at 10.75um. An unidentified absorption feature at 13.7um is detected in many of the starbursts. We use the fine-structure lines to derive the abundance of neon and sulfur for 14 objects where the HI 7-6 line is detected. We further use the molecular hydrogen lines to sample the properties of the warm molecular gas. Several basic diagrams characterizing the properties of the sample are also shown. We have combined the spectra of all the pure starburst objects to create a high signal-to-noise ratio template, which is available to the community.