Source, in full
On refusing requests to write his memoirs, as quoted in Gentlemen of Virginia (1961) page 188 by Marshall William Fishwick; also cited as possibly apocryphal in The Oxford Dictionary of Quotations (2004) edited by Elizabeth M. Knowles