Source, in full
in Henry T. Riley, A Dictionary of Latin and Greek Quotations, Proverbs, Maxims and Mottos (1876), p. 138. This is a translation of Ovid's "gratis paenitet esse probum" from his Ex Ponto, book 2, chapter 3. The translation in Bartlett's Familiar Quotations, 15th ed., p. 114 (1980) reads: "It is annoying to be honest to no purpose".