Source, in full
Attributed in William R. Cox, "Maj.-Gen. Stephen D. Ramseur, His Life and Character", An Address before the Ladies' Memorial Association of Raleigh, NC, May 7, 1891. Southern Historical Society Papers, vol. 18 (1890) p. 255
Notes from Wikiquote
Also reported by Gen. Basil W. Duke and Gen. Richard Taylor. Often erroneously reported as "Git thar fustest with the most mostest." In The Quote Verifier: Who Said What, Where, and When (2006) by Ralph Keyes, p. 272, the phrase he used has also been reported to have been "I always make it a rule to get there first with the most men" and "I just took the short cut and got there first with the most men."