We present a catalog of high proper motion (HPM) stars detected in the foreground of central parts of the Magellanic Clouds. The Catalog contains 2161 objects in the 4.5 square degree area towards the LMC, and 892 HPM stars in the 2.4 square degree area towards the SMC. The Catalog is based on observations collected during four years of the OGLE-II microlensing survey. The Difference Image Analysis (DIA) of the images provided candidate HPM stars with proper motion as small as 4mas/yr. These appeared as pseudo-variables, and were all measured astrometrically on all CCD images, providing typically about 400 data points per star. The reference frame was defined by the majority of background stars, most of them members of the Magellanic Clouds. The reflex motion due to solar velocity with respect to the local standards of rest is clearly seen. The largest proper motion in our sample is 363mas/yr. Parallaxes were measured with errors smaller than 20% for several stars.