Yale University has removed three portraits of controversial slavery advocate John Calhoun from a residential college named in his honor and is contemplating changing the name of the college.

The decision arose out of tensions at the Ivy League school when racially-insensitive Halloween costumes led to racially-charged debates.

Calhoun, a prominent U.S. statesman who graduated from Yale in 1804, was an avid defender of the slave plantation system. The Southern politician served in the U.S. executive branch—twice as vice president under John Quincy Adams and Andrew Jackson, as U.S. secretary of war and secretary of state; and he also served in the legislative branch, representing South Carolina in the U.S. Senate, according to

Yale’s Calhoun College was named after the unbridled White supremacist in 1932. Since then, three of his likenesses have adorned walls of the building—one in the dining hall and two in the college master’s house.

Calhoun Master Julia Adams in mid-January announced plans to remove the three portraits in an effort to address students concern about the racist symbolism of the artwork, she told the Yale Daily News. She also said the space will remain empty until the school resolves the dispute over the name of the college.

“An empty space at this point actually befits the stage that we’re at in the conversation,” she said. “We are waiting for a decision now.”

According to the Daily News, a majority of students and faculty interviewed supported Adams’ removal of the portraits.

“To adorn public spaces at Yale with portraits is to place a symbolic value on those portraits and, in the absence of critical engagement and critique, to tacitly endorse the legacy of those portrayed,” Classics Director of Undergraduate Studies Emily Greenwood, told the publication. “I welcome Professor Adams’ proactive decision to remove these portraits that cause disquiet to many residents of the college.”