Stars of spectral types O and B produce neutron stars (NSs) after supernova explosions. Most of NSs are strongly magnetized including normal radio pulsars with B {prop.to} 1012 G and magnetars with B {prop.to} 10^14^ G. A fraction of 7-12 per cent of massive stars are also magnetized with B {prop.to} 103 G and some are weakly magnetized with B {prop.to} 1 G. It was suggested that magnetic fields of NSs could be the fossil remnants of magnetic fields of their progenitors. This work is dedicated to study this hypothesis. First, we gather all modern precise measurements of surface magnetic fields in O, B, and A stars. Secondly, we estimate parameters for lognormal distribution of magnetic fields in B stars and found {mu}B = 2.83 +/- 0.1 log10 (G), {sigma}B = 0.65 +/- 0.09 for strongly magnetized and {mu}B = 0.14 +/- 0.5 log10 (G), {sigma} = 0.7_-0.27_^+0.57^ for weakly magnetized. Third, we assume that the magnetic field of pulsars and magnetars have 2.7-dex difference in magnetic fields and magnetars represent 10 per cent of all young NSs and run population synthesis. We found that it is impossible to simultaneously reproduce pulsars and magnetars populations if the difference in their magnetic fields is 2.7 dex. Therefore, we conclude that the simple fossil origin of the magnetic field is not viable for NSs.