Source, in full
Motto of United States Magazine and Democratic Review. First used in introductory essay by editor John L. O'Sullivan in the premier issue (October 1837, p. 6). Attributed to Jefferson by Henry David Thoreau, this statement is cited in his essay on civil disobedience, but the quote has not been found in Jefferson's own writings. It is also commonly attributed to Thomas Paine, perhaps because of its similarity in theme to many of his well-documented expressions such as "Society in every state is a blessing, but government even in its best state is but a necessary evil; in its worst state an intolerable one."
Variants
"That government is best which governs least"; reported in Paul F. Boller, Jr., and John George, They Never Said It: A Book of Fake Quotes, Misquotes, & Misleading Attributions (1989), p. 56