(CNN) - Artur Davis, the former four-term Democratic congressman from Alabama, said Wednesday that Vice President Joe Biden was propagating "racial viciousness" when he said Mitt Romney's regulatory policies would "put y'all back in chains."
Biden made the comments Tuesday to a predominantly black crowd in Danville, Virginia, a city with a long history of racial tension. In May Davis announced he was switching to the GOP, leaving the door open to a future political bid as a Republican.
"[Romney] is going to let the big banks once again write their own rules, unchain Wall Street," Biden said. "He is going to put y'all back in chains."
Davis, who is black, said the comment smacked of a type of divisiveness he said was all too common in the South.
"It brought back memories for me," Davis told Wolf Blitzer on CNN's "The Situation Room." "It brought back memories of these Democratic politicians in the South, who think they can go before crowds and say one thing and nobody else will hear it, and they'll somehow get a cheer in the room and that they can blithely go on about their business."
Saying Biden went to a place he "never should have gone" in his remarks, Davis called the vice president's words insulting.
"It's a divisive tactic that's insulting to African Americans," Davis said. "It's insulting to the American people. It's insulting to the legacy that he used to build up as an orator who used to know how to inspire people instead of strike fear in people's heart."
Davis, who campaigned for Mitt Romney Wednesday in Virginia, may be best known for seconding Obama's nomination at the Democratic National Convention in 2008, when he served as an Obama campaign co-chairman.
Davis said he had hoped Obama's presidency would make a huge dent in race relations, as well as move the Democratic Party further to the center. Biden's comments, Davis said, should embarrass the president.
"It ought to embarrass President Obama," he said. "President Obama has talked so movingly about our country moving beyond race, and his own vice president makes this kind of comment yesterday? It was wrong and the president ought to be embarrassed by it, and the president ought to say it was wrong."
On Tuesday, Obama's deputy campaign manager Stephanie Cutter attempted to dispel the controversy around Biden's remarks, comparing his words to Republicans who have called for the "unshackling" of the private sector from regulations in recent months.
"Since then, the Vice President has often used a similar metaphor to describe the need to 'unshackle' the middle class," Cutter said in a statement. "Today's comments were a derivative of those remarks, describing the devastating impact letting Wall Street write its own rules again would have on middle class families."
Cutter added that they "find the Romney campaign's outrage over the Vice President's comments today hypocritical, particularly in light of their own candidate's stump speech questioning the President's patriotism."
Davis wrote off that explanation as "spin" in "The Situation Room" Wednesday, saying he gave Obama's team "credit for creativity."
"But I happen to have spoken to a few African-American audiences in my time, represented a predominantly African-American district," Davis said. "I know what Joe Biden was doing yesterday, and every black person in the room knew who the 'y'all' was, they knew what the chains were about, and they knew what the metaphor was."