Using the Lyman Dropout technique, we identify 148 candidate radio sources at z ~> 4-7 from the 887.5 MHz Australian Square Kilometre Array Pathfinder (ASKAP) observations of the GAMA23 field. About 112 radio sources are currently known beyond redshift z ~ 4. However, simulations predict that hundreds of thousands of radio sources exist in that redshift range, many of which are probably in existing radio catalogues, but do not have measured redshifts, either because their optical emission is too faint or because of the lack of techniques that can identify candidate high-redshift radio sources (HzRSs). Our study addresses these issues using the Lyman Dropout search technique. This newly built sample probes radio luminosities that are 1-2 orders of magnitude fainter than known radio-active galactic nuclei (AGN) at similar redshifts, thanks to ASKAP's sensitivity. We investigate the physical origin of radio emission in our sample using a set of diagnostics: (i) radio luminosity at 1.4 GHz, (ii) 1.4 GHz to 3.4 {mu}m flux density ratio, (iii) Far-IR detection, (iv) WISE colour, and (v) SED modelling. The radio/IR analysis has shown that the majority of radio emission in the faint and bright end of our sample's 887.5 MHz flux density distribution originates from AGN activity. Furthermore, ~10 per cent of our sample are found to have a 250 {mu}m detection, suggesting a composite system. This suggests that some high-z radio-AGNs are hosted by SB galaxies in contrast to low-z radio- AGNs, which are usually hosted by quiescent elliptical galaxies.