We analyze the population of periodic variable stars in the open cluster NGC3766 based on a 7-year multiband monitoring campaign conducted on the 1.2m Swiss Euler telescope at La Silla, Chili. The data reduction, light curve cleaning, and period search procedures, combined with the long observation time line, allowed us to detect variability amplitudes down to the mmag level. The variability properties were complemented with the positions in the color-magnitude and color-color diagrams to classify periodic variable stars into distinct variability types. We find a large population (36 stars) of new variable stars between the red edge of slowly pulsating B (SPB) stars and the blue edge of delta Sct stars, a region in the Hertzsprung-Russell (HR) diagram where no pulsation is predicted to occur based on standard stellar models. The bulk of their periods ranges from 0.1 to 0.7d, with amplitudes between 1 and 4mmag for the majority of them. About 20% of stars in that region of the HR diagram are found to be variable, but the number of members of this new group is expected to be higher, with amplitudes below our mmag detection limit. The properties of this new group of variable stars are summarized and arguments set forth in favor of a pulsation origin of the variability, with g-modes sustained by stellar rotation. Potential members of this new class of low-amplitude periodic (most probably pulsating) A and late-B variables in the literature are discussed. We additionally identify 16 eclipsing binary, 13 SPB, 14 delta Sct, and 12 gamma Dor candidates, as well as 72 fainter periodic variables. All are new discoveries. We encourage searching for this new class of variables in other young open clusters, especially in those hosting a rich population of Be stars.