We quantify the degree of lopsidedness for a sample of 149 galaxies observed in the near-infrared from the Ohio State University Bright Galaxy Survey (OSUBGS, Eskridge et al. 2002, Cat. <J/ApJS/143/73>) sample, and try to explain the physical origin of the observed disk lopsidedness. We confirm previous studies, but for a larger sample, that a large fraction of galaxies have significant lopsidedness in their stellar disks, measured as the Fourier amplitude of the m=1 component normalised to the average or m=0 component in the surface density. Late-type galaxies are found to be more lopsided, while the presence of m=2 spiral arms and bars is correlated with disk lopsidedness. We also show that the m=1 amplitude is uncorrelated with the presence of companions. Numerical simulations were carried out to study the generation of m=1 via different processes: galaxy tidal encounters, galaxy mergers, and external gas accretion with subsequent star formation. These simulations show that galaxy interactions and mergers can trigger strong lopsidedness, but do not explain several independent statistical properties of observed galaxies. To explain all the observational results, it is required that a large fraction of lopsidedness results from cosmological accretion of gas on galactic disks, which can create strongly lopsided disks when this accretion is asymmetrical enough.