Classification of stars, by comparing their optical spectra to a few dozen spectral standards, has been a workhorse of observational astronomy for more than a century. Here, we extend this technique by compiling a library of optical spectra of 404 touchstone stars observed with Keck/HIRES by the California Planet Search. The spectra have high resolution (R~60000), high signal-to-noise ratio (S/N~150/pixel), and are registered onto a common wavelength scale. The library stars have properties derived from interferometry, asteroseismology, LTE spectral synthesis, and spectrophotometry. To address a lack of well-characterized late-K dwarfs in the literature, we measure stellar radii and temperatures for 23 nearby K dwarfs, using modeling of the spectral energy distribution and Gaia parallaxes. This library represents a uniform data set spanning the spectral types ~M5-F1 (T_eff_~3000-7000K, R_*_~0.1-16R_{Sun}_). We also present "Empirical SpecMatch" (SpecMatch-Emp), a tool for parameterizing unknown spectra by comparing them against our spectral library. For FGKM stars, SpecMatch-Emp achieves accuracies of 100K in effective temperature (T_eff_), 15% in stellar radius (R_*_), and 0.09dex in metallicity ([Fe/H]). Because the code relies on empirical spectra it performs particularly well for stars ~K4 and later, which are challenging to model with existing spectral synthesizers, reaching accuracies of 70K in T_eff_, 10% in R_*_, and 0.12dex in [Fe/H]. We also validate the performance of SpecMatch-Emp, finding it to be robust at lower spectral resolution and S/N, enabling the characterization of faint late-type stars. Both the library and stellar characterization code are publicly available.