1806, originally was a military drill to teach balance; "to stand on each leg alternately and swing the other back and forth" (which, presumably, reminded someone of a goose's way of walking); in reference to "marching without bending the knees" (as in Nazi military reviews) it apparently is first recorded 1916. As a verb by 1854.
A straight-legged style of military marching used by the armies of several nations, but associated particularly with the army of Germany under the Nazis.
Note: The term is sometimes used to suggest the unthinking loyalty of followers or soldiers: “Brown has a goose-step mentality.”