Murphy became editor of the Baltimore Afro-American due to the poor health of his father in 1918. Later, after his father passed away in 1922 Murphy became the leader of one of the most influential African American publications in the United States. At the peak of its circulation the Baltimore Afro-American reached over 200,000 people, and Murphy helped the paper grow in size so that by the end of his time at the paper in 1961, it had expanded to cover Newark, Philadelphia, Washington D.C., and Richmond, Virginia.
Carl Murphy was also a member of the board of directors for the National Association for the Advancement of Colored People, being appointed in 1931, and fought for an end to legal discrimination. He was also active in debates on both the local and national levels. He was also president of the National Newspaper Publishers’ Association in 1954 and 1955 and was dedicated to the improvement of African American education. Murphy was married to Vashti Turley Murphy, co-founder of Delta Sigma Theta Sorority. Murphy died on February 22, 1967 leaving the paper to go down the Murphy line to his great nephew John Oliver Jr.