Gaia's precision astrometry allows systematic identification of optically selected subkiloparsec dual active galactic nuclei (AGNs), off-nucleus AGNs, and small-scale lensed quasars by "varstrometry"-where variability-induced astrometric jitter, i.e., temporal displacements of photocenter in unresolved sources, can be reasonably well detected or constrained. This approach extends systematic searches for small-scale (>=mas) dual and off-nucleus AGNs to the poorly explored regime between ~10pc and ~1kpc, with Gaia's full sky coverage and depth to G~21. We outline the general principles of this method and calculate the expected astrometric signals from the full time series of photocenter measurements and light curves. We demonstrate the feasibility of varstrometry by using Gaia DR2 data on a sample of variable pre-main-sequence stars with known close companions. We find that extended host galaxies have a significant impact on the accuracy of astrometric and photometric variability in Gaia DR2, a situation to be improved in future Gaia releases. Using spectroscopically confirmed Sloan Digital Sky Survey quasars, we present several examples of candidate subkiloparsec off-nucleus or dual AGNs selected from Gaia DR2. We discuss the merits and limitations of this method and a follow-up strategy for promising candidates. We highlight Gaia's potential of systematically discovering and characterizing the subkiloparsec off-nucleus and dual AGN population in the entire optical sky.