adjective and noun suffix, "having to do with, characterized by, tending to, place for," from Middle English -orie, from Old North French -ory, -orie (Old French -oir, -oire), from Latin -orius, -oria, -orium.
Latin adjectives in -orius, according to "An Etymological Dictionary of the French Language," tended to "indicate a quality proper to the action accomplished by the agent; as oratorius from orator; laudatorius from laudator. The neuter of these adjectives was early employed as a substantive, and usually denoted the place of residence of the agent or the instrument that he uses; as praetorium from praetor; dormitorium from dormitor; auditorium, dolatorium.
"These newer words, already frequent under the Empire, became exceedingly numerous at a later time, especially in ecclesiastical and scholastic Latin; as purgatorium, refectorium, laboratorium, observatorium, &c." [transl. G.W. Kitchin, Oxford, 1878]