We describe a blind uniform search for thermonuclear burst oscillations (TBOs) in the majority of Type I bursts observed by the Rossi X-ray Timing Explorer (RXTE) (2118 bursts from 57 neutron stars). We examined 2-2002Hz power spectra from the Fourier transform in sliding 0.5-2s windows, using fine-binned light curves in the 2-60keV energy range. The significance of the oscillation candidates was assessed by simulations which took into account light-curve variations, dead time, and the sliding time windows. Some of our sources exhibited multi-frequency variability at <~15Hz that cannot be readily removed with light-curve modeling and may have an astrophysical (non-TBO) nature. Overall, we found that the number and strength of potential candidates depends strongly on the parameters of the search. We found candidates from all previously known RXTE TBO sources, with pulsations that had been detected at similar frequencies in multiple independent time windows, and discovered TBOs from SAXJ1810.8-2658. We could not confirm most previously reported tentative TBO detections or identify any obvious candidates just below the detection threshold at similar frequencies in multiple bursts. We computed fractional amplitudes of all TBO candidates and placed upper limits on non-detections. Finally, for a few sources we noted a small excess of candidates with powers comparable to fainter TBOs, but appearing in single independent time windows at random frequencies. At least some of these candidates may be noise spikes that appear interesting due to selection effects. The potential presence of such candidates calls for extra caution if claiming single-window TBO detections.