We present the optical identifications and a multi-band catalogue of a sample of 478 X-ray sources detected in the XMM-Newton and Chandra surveys of the central 0.6deg^2^ of the ELAIS-S1 field. The most likely optical/infrared counterpart of each X-ray source was identified using the chance coincidence probability in the R and IRAC 3.6 micron bands.This method was complemented by the precise positions obtained through Chandra observations. We were able to associate a counterpart to each X-ray source in the catalogue. Approximately 94% of them are detected in the R band, while the remaining are blank fields in the optical down to R~24.5, but have a near-infrared counterpart detected by IRAC within 6 arcsec from the XMM-Newton centroid. The multi-band catalogue, produced using the positions of the identified optical counterparts, contains photometry in ten photometric bands, from B to the MIPS 24 micron band. The spectroscopic follow-up allowed us to determine the redshift and classification for 237 sources (~50% of the sample) brighter than R=24. The spectroscopic redshifts were complemented by reliable photometric redshifts for 68 sources. We classified 47% of the sources with spectroscopic redshift as broad-line active galactic nuclei (BL AGNs) with z=0.1-3.5, while sources without broad-lines (NOT BL AGNs) are about 46% of the spectroscopic sample and are found up to z=2.6. The remaining fraction is represented by extended X-ray sources and stars. We spectroscopically identified 11 type 2 QSOs among the sources with X/O>8, with redshift between 0.9 and 2.6, high 2-10keV luminosity (logLx>=43.8erg/s) and hard X-ray colors suggesting large absorbing columns at the rest frame (logN_H_ up to 23.6cm^-2^). BL AGNs show on average blue optical-to-near-infrared colors, softer X-ray colors and X-ray-to-optical colors typical of optically selected AGNs. Conversely, narrow-line sources show redder optical colors, harder X-ray flux ratio and span a wider range of X-ray-to-optical colors. On average the Spectral Energy Distributions (SEDs) of high-luminosity BL AGNs resemble the power-law typical of unobscured AGNs. The SEDs of NOT BLAGNs are dominated by the galaxy emission in the optical/near-infrared, and show a rise in the mid-infrared which suggests the presence of an obscured active nucleus. We study the infrared-to-optical colors and near-infrared SEDs to infer the properties of the AGN host galaxies.