Emission from X-ray binaries (XRBs) is a major component of the total X-ray luminosity of normal galaxies, so X-ray studies of high-redshift galaxies allow us to probe the formation and evolution of XRBs on very long timescales (~10Gyr). In this paper, we present results from large-scale population synthesis models of binary populations in galaxies from z=0 to ~20. We use as input into our modeling the Millennium II Cosmological Simulation and the updated semi-analytic galaxy catalog by Guo et al. (2011MNRAS.413..101G) to self-consistently account for the star formation history (SFH) and metallicity evolution of each galaxy. We run a grid of 192 models, varying all the parameters known from previous studies to affect the evolution of XRBs. We use our models and observationally derived prescriptions for hot gas emission to create theoretical galaxy X-ray luminosity functions (XLFs) for several redshift bins. Models with low common envelope efficiencies, a 50% twins mass ratio distribution, a steeper initial mass function exponent, and high stellar wind mass-loss rates best match observational results from Tzanavaris & Georgantopoulos, though they significantly underproduce bright early-type and very bright (L_x_>10^41^) late-type galaxies. These discrepancies are likely caused by uncertainties in hot gas emission and SFHs, active galactic nucleus contamination, and a lack of dynamically formed low-mass XRBs. In our highest likelihood models, we find that hot gas emission dominates the emission for most bright galaxies. We also find that the evolution of the normal galaxy X-ray luminosity density out to z=4 is driven largely by XRBs in galaxies with X-ray luminosities between 10^40^ and 10^41^erg/s.