We present an atlas of spectra of O- and B-type stars, obtained with the Short Wavelength Spectrometer (SWS) during the Post-Helium program of the Infrared Space Observatory (ISO). This program is aimed at extending the Morgan & Keenan (1973ARA&A..11...29M) classification scheme into the near-infrared. Later type stars will be discussed in a separate publication. The observations consist of 57 SWS Post-Helium spectra from 2.4 to 4.1{mu}m, supplemented with 10 spectra acquired during the nominal mission with a similar observational setting. For B-type stars, this sample provides ample spectral coverage in terms of subtype and luminosity class. For O-type stars, the ISO sample is coarse and therefore is complemented with 8 UKIRT L'-band observations. In terms of the presence of diagnostic lines, the L'-band is likely the most promising of the near-infrared atmospheric windows for the study of the physical properties of B stars. Specifically, this wavelength interval contains the Br{alpha}, Pf{gamma}, and other Pfund lines which are probes of spectral type, luminosity class and mass loss. Here, we present simple empirical methods based on the lines present in the 2.4 to 4.1{mu}m interval that allow the determination of i) the spectral type of B dwarfs and giants to within two subtypes; ii) the luminosity class of B stars to within two classes; iii) the mass-loss rate of O stars and B supergiants to within 0.25dex.