Souther was born in Detroit, Michigan and raised in Amarillo, Texas.
After moving to Los Angeles, California, in the late 1960s, he met musician and songwriter, Glenn Frey and the two became roommates and musical collaborators. Souther and Frey formed a folk duo called Longbranch Pennywhistle. Their lone album was released in 1970 on Jimmy Bowen's Amos Records.
Souther married singer Alexandra Sliwin in March 1969, but divorced in 1972. He dated Linda Ronstadt and Stevie Nicks in the 1970s. Judee Sill's song "Jesus Was a Crossmaker" was written for Souther, who she says broke her heart after a short affair. In December 2002, Souther moved from the Hollywood Hills, California, to Nashville, Tennessee. He remarried in 2004 but divorced a second time in 2010.