If you are out to describe the truth, leave elegance to the tailor.

Earliest attribution located is The Yogi and the Commissar by Arthur Koestler (1945), p. v. Koestler prefaces it with "My comfort is what Einstein said when somebody reproached him with the suggestion that his formula of gravitation was longer and more cumbersome than Newton's formula in its elegant simplicity". This is actually a variant of a quote Einstein attributed to Ludwig Boltzmann; in the Preface to his Relativity—The Special and General Theory (1916), Einstein wrote: "I adhered scrupulously to the precept of that brilliant theoretical physicist L. Boltzmann, according to whom matters of elegance ought to be left to the tailor and to the cobbler." (reprinted in the 2007 book A Stubbornly Persistent Illusion: The Essential Scientific Works of Albert Einstein edited by Stephen Hawking, p. 128)