We present Neil Gehrels Swift Observatory (hereafter Swift) observations of 31 sources from the SMS4 catalog, a sample of 137 bright radio sources originally designed to extend the well-studied 3CRR radio sample to the Southern Hemisphere. All these sources had no Chandra or XMM-Newton observations: 24 of these were observed with Swift through a dedicated proposal in 2015, and data for the remaining seven were retrieved from the Swift archive. The reduction and analysis of data collected by the Swift X-ray Telescope (XRT) led to 20 detections in the 0.3-10 keV band. We provide details of the X-ray emission in this band for these 20 detections, as well as upper limits for the remaining 11 SMS4 sources. When statistics allowed, we investigated the extent of the X-ray emission and the hardness ratio, and we carried out a spectral analysis. We matched the 20 X-ray-detected sources with infrared (AllWISE, CatWISE2020) and optical (GSC 2.3.2, DES DR2) catalogs to establish associations with infrared and optical sources and compared our results with previously published counterparts in these bands. Requiring a detection in both the infrared and optical bands to establish a candidate counterpart for our X-ray detections, we obtain reliable counterparts for 18 sources, while the remaining two sources need further investigation to establish firm identifications. In the infrared, we confirm 12 previously established counterparts and provide six new candidates. In the optical, we find agreement with 13 previously established counterparts, while we provide an alternative candidate for five SMS4 sources. We find that ~35% of all the SMS4 sources lie below the lower limit of 10.9Jy for the flux density at 178MHz established for the 3CRR sample, at variance with the values extrapolated using measurements at higher frequencies. Therefore, for future studies where flux-density-limited samples are needed, we encourage the use of the more recent G4Jy sample. We present the list of 56 SMS4 sources that in 2022 March remain to be observed in the X-rays with narrow-field instruments, to highlight interesting sources and organize further observational campaigns to achieve complete X-ray coverage for the whole SMS4 in the forthcoming years.