We constrain the minimum variability timescales for 938 gamma-ray bursts (GRBs) observed by the Fermi/Gamma-ray Burst Monitor instrument prior to 2012 July 11. The tightest constraints on progenitor radii derived from these timescales are obtained from light curves in the hardest energy channel. In the softer bands --or from measurements of the same GRBs in the hard X-rays from Swift-- we show that variability timescales tend to be a factor of two to three longer. Applying a survival analysis to account for detections and upper limits, we find median minimum timescale in the rest frame for long-duration and short-duration GRBs of 45 and 10ms, respectively. Less than 10% of GRBs show evidence for variability on timescales below 2ms. These shortest timescales require Lorentz factors >~400 and imply typical emission radii R~1x10^14^cm for long-duration GRBs and R~3x10^13^cm for short-duration GRBs. We discuss implications for the GRB fireball model and investigate whether or not GRB minimum timescales evolve with cosmic time.