Source, in full
As quoted in Innerspace : Introduction to Kabbalah, Meditation and Prophecy (1990), by Aryeh Kaplan, p. 160; This expression is said to have originated when he was five years old and asked his father "Where is God?" to which his father answered: "God is everywhere!" Rebbe then responded "No, I think God is only where you let Him in."
Notes from Wikiquote
Where is God to be found? In the place where He is given entry. | As quoted in The Sayings of Menahem Mendel of Kotsk (1995) by Simcha Raz and Edward Levin, p. 10 <!-- | also The Challenge of the Soul : A Guide for the Spiritual Warrior (2009) by Niles Elliot Goldstein --> | God is where you let Him in. | As quoted in Everyday Kabbalah: A Practical Guide to Jewish Meditation, Healing, and Personal Growth (1998)