Stellar light curves are well known to encode physical stellar properties. Precise, automated, and computationally inexpensive methods to derive physical parameters from light curves are needed to cope with the large influx of these data from space-based missions such as Kepler and TESS. Here we present a new methodology that we call "The Swan", a fast, generalizable, and effective approach for deriving stellar surface gravity (logg) for main-sequence, subgiant, and red giant stars from Kepler light curves using local linear regression on the full frequency content of Kepler long-cadence power spectra. With this inexpensive data-driven approach, we recover logg to a precision of ~0.02dex for 13822 stars with seismic logg values between 0.2 and 4.4dex and ~0.11dex for 4646 stars with Gaia-derived logg values between 2.3 and 4.6dex. We further develop a signal-to-noise metric and find that granulation is difficult to detect in many cool main-sequence stars (Teff<~5500K), in particular K dwarfs. By combining our logg measurements with Gaia radii, we derive empirical masses for 4646 subgiant and main-sequence stars with a median precision of ~7%. Finally, we demonstrate that our method can be used to recover logg to a similar mean absolute deviation precision for a TESS baseline of 27days. Our methodology can be readily applied to photometric time series observations to infer stellar surface gravities to high precision across evolutionary states.