Source, in full
The phrase has been used commonly for several decades, and there may be earlier sources. In a 1992 Business North Carolina article, Dooley noted about being an NFL referee that "If you're wrong, everybody in the world will tell you. If you're right, nobody cares. I'm not always right, but I'm never in doubt." Cf. the Free Library article.