We present the classification of 197 point sources observed with the Infrared Spectrograph in the SAGE-Spec Legacy programme on the Spitzer Space Telescope. We introduce a decision-tree method of object classification based on infrared spectral features, continuum and spectral energy distribution shape, bolometric luminosity, cluster membership and variability information, which is used to classify the SAGE-Spec sample of point sources. The decision tree has a broad application to mid-infrared spectroscopic surveys, where supporting photometry and variability information are available. We use these classifications to make deductions about the stellar populations of the Large Magellanic Cloud and the success of photometric classification methods. We find 90 asymptotic giant branch (AGB) stars, 29 young stellar objects, 23 post-AGB objects, 19 red supergiants, eight stellar photospheres, seven background galaxies, seven planetary nebulae, two HII regions and 12 other objects, seven of which remain unclassified.