Old English gat "she-goat," from Proto-Germanic *gaitaz (cf. Old Saxon get, Old Norse geit, Danish gjed, Middle Dutch gheet, Dutch geit, Old High German geiz, German Geiß, Gothic gaits "goat"), from PIE *ghaidos "young goat," also "play" (cf. Latin hædus "kid").
The word for "male goat" in Old English was bucca (see buck (n.)) until late 1300s shift to he-goat, she-goat (Nanny goat is 18c., billy goat 19c.). Meaning "licentious man" is attested from 1670s. To get (someone's) goat is from 1910, perhaps with notion of "to steal a goat mascot from a racehorse," or from French prendre sa chèvre "take one's source of milk."
(1.) Heb. 'ez, the she-goat (Gen. 15:9; 30:35; 31:38). This Hebrew word is also used for the he-goat (Ex. 12:5; Lev. 4:23; Num. 28:15), and to denote a kid (Gen. 38:17, 20). Hence it may be regarded as the generic name of the animal as domesticated. It literally means "strength," and points to the superior strength of the goat as compared with the sheep. (2.) Heb. 'attud, only in plural; rendered "rams" (Gen. 31:10,12); he-goats (Num. 7:17-88; Isa. 1:11); goats (Deut. 32:14; Ps. 50:13). They were used in sacrifice (Ps. 66:15). This word is used metaphorically for princes or chiefs in Isa. 14:9, and in Zech. 10:3 as leaders. (Comp. Jer. 50:8.) (3.) Heb. gedi, properly a kid. Its flesh was a delicacy among the Hebrews (Gen. 27:9, 14, 17; Judg. 6:19). (4.) Heb. sa'ir, meaning the "shaggy," a hairy goat, a he-goat (2 Chr. 29:23); "a goat" (Lev. 4:24); "satyr" (Isa. 13:21); "devils" (Lev. 17:7). It is the goat of the sin-offering (Lev. 9:3, 15; 10:16). (5.) Heb. tsaphir, a he-goat of the goats (2 Chr. 29:21). In Dan. 8:5, 8 it is used as a symbol of the Macedonian empire. (6.) Heb. tayish, a "striker" or "butter," rendered "he-goat" (Gen. 30:35; 32:14). (7.) Heb. 'azazel (q.v.), the "scapegoat" (Lev. 16:8, 10,26). (8.) There are two Hebrew words used to denote the undomesticated goat:, _Yael_, only in plural mountain goats (1 Sam. 24:2; Job 39:1; Ps.104:18). It is derived from a word meaning "to climb." It is the ibex, which abounded in the mountainous parts of Moab. And _'akko_, only in Deut. 14:5, the wild goat. Goats are mentioned in the New Testament in Matt. 25:32,33; Heb. 9:12,13, 19; 10:4. They represent oppressors and wicked men (Ezek. 34:17; 39:18; Matt. 25:33). Several varieties of the goat were familiar to the Hebrews. They had an important place in their rural economy on account of the milk they afforded and the excellency of the flesh of the kid. They formed an important part of pastoral wealth (Gen. 31:10, 12;32:14; 1 Sam. 25:2).