Source, in full
This is attributed to Wellington as a statement to an unidentified woman at a reception in Vienna, who had apologized for the rudeness of some French officers who had turned their backs on him when he entered, as quoted in The Guinness Book of Military Anecdotes (1992), by Geoffrey Regan, page 27
Notes from Wikiquote
This is attributed to Wellington as a statement to King Louis XVIII at a ball in the spring of 1814, as quoted in "Anecdotes of Wellington" at The Wellington Society of Madrid
Variants
'Tis of no matter, your Highness, I have seen their backs before.