Interstellar dust permeates our Galaxy and plays an important role in many physical processes in the diffuse and dense regions of the interstellar medium (ISM). High-resolution X-ray spectroscopy, coupled with modelling based on laboratory dust measurements, provides a unique probe for investigating the interstellar dust properties along our line of sight towards Galactic X-ray sources. Here, we focus on the oxygen content of the ISM through its absorption features in the X-ray spectra. To model the dust features, we perform a laboratory experiment using the electron microscope facility located at the University of Cadiz in Spain, where we acquire new laboratory data in the oxygen K-edge. We study 18 dust samples of silicates and oxides with different chemical compositions. The laboratory measurements are adopted for our astronomical data analysis. We carry out a case study on the X-ray spectrum of the bright low-mass X-ray binary Cygnus X-2, observed by XMM-Newton. We determine different temperature phases of the ISM and parameterise oxygen in both gas (neutral and ionised) and dust form. We find Solar abundances of oxygen along the line of sight towards the source. Due to both the relatively low depletion of oxygen into dust form and the shape of the oxygen cross section profiles, it is challenging to determine the precise chemistry of interstellar dust. However, silicates provide an acceptable fit. Finally, we discuss the systematic discrepancies in the atomic (gaseous phase) data of the oxygen edge spectral region using different X-ray atomic databases as well as consider future prospects for studying the ISM with the Arcus concept mission.