We present a new automated method to identify instrumental features masquerading as small, long-period planets in the Kepler planet candidate catalog. These systematics, mistakenly identified as planet transits, can have a strong impact on occurrence rate calculations because they cluster in a region of parameter space where Kepler's sensitivity to planets is poor. We compare individual transit-like events to a variety of models of real transits and systematic events and use a Bayesian information criterion to evaluate the likelihood that each event is real. We describe our technique and test its performance on simulated data. Results from this technique are incorporated in the Kepler Q1-Q17 DR24 planet candidate catalog of Coughlin et al. (2016, Cat. J/ApJS/224/12)