Source, in full
This saying was not originated but popularized by Truman after he publicly used it in 1952. It was soon credited to his aide Harry H. Vaughan in Time (28 April 1952) but apparently originated with a Missouri colleague of Truman, Eugene "Buck" Purcell, according to The Quote Verifier: Who Said What, Where, And When (2006) by Ralph Keyes. Truman himself later made reference to his popularization of the remark in his book Mr. Citizen (1960), p. 229
Notes from Wikiquote
There has been a lot of talk lately about the burdens of the Presidency. Decisions that the President has to make often affect the lives of tens of millions of people around the world, but that does not mean that they should take longer to make. Some men can make decisions and some cannot. Some men fret and delay under criticism. I used to have a saying that applies here, and I note that some people have picked it up, "If you can't stand the heat, get out of the kitchen."