It is for the Prime Minister to decide whether he expressed himself appropriately in the Commons. It is up to him as to whether he wants to annoy 51% of the population.

Notes from Wikiquote

In the House of Commons, David Cameron had told the then shadow Treasury chief secretary to "calm down, dear" at Prime Minister's Questions.