A recent study of a small sample of X-ray binaries (XRBs) suggests a significant softening of spectra of neutron star (NS) binaries as compared to black hole (BH) binaries in the luminosity range 10^34^-10^37^erg/s. This softening is quantified as an anticorrelation between the spectral index and the 0.5-10keV X-ray luminosity. We extend the study to significantly lower luminosities (i.e., ~a few x10^30^erg/s) for a larger sample of XRBs. We find evidence for a significant anticorrelation between the spectral index and the luminosity for a group of NS binaries in the luminosity range 10^32^-10^33^erg/s. Our analysis suggests a steep slope for the correlation i.e., -2.12+/-0.63. In contrast, BH binaries do not exhibit the same behavior. We examine the possible dichotomy between NS and BH binaries in terms of a Comptonization model that assumes a feedback mechanism between an optically thin hot corona and an optically thick cool source of soft photons. We gauge the NS-BH dichotomy by comparing the extracted corona temperatures, Compton-y parameters, and the Comptonization amplification factors: the mean temperature of the NS group is found to be significantly lower than the equivalent temperature for the BH group. The extracted Compton-y parameters and the amplification factors follow the theoretically predicted relation with the spectral index.