Khan was born in a Punjabi Muslim family in Faisalabad in 1948, shortly after the partition of India in 1947 during which his family had migrated to Pakistan from their native city of Jalandhar in East Punjab, British India (now in Punjab, India). Before partition, his family lived in their ancestral house at Basti Sheikh, Jalandhar. He was the fifth child and first son of Fateh Ali Khan, a musicologist, vocalist, instrumentalist, and qawwal. Khan's family, which included four older sisters and a younger brother, Farrukh Fateh Ali Khan, grew up in central Faisalabad. The tradition of qawwali in the family had passed down through successive generations for almost 600 years. Initially, his father did not want Khan to follow the family's vocation. He had his heart set on Nusrat choosing a much more respectable career path and becoming a doctor or engineer, because he felt Qawwali artists had low social status. However, Khan showed such an aptitude for and interest in Qawwali, that his father finally relented. He began by learning the tabla before moving on to vocals. In 1964, Khan's father died, leaving his musical education under the supervision of his paternal uncles, Mubarak Ali Khan and Salamat Ali Khan. He is the uncle of singer Rahat Fateh Ali Khan.