Specter, who died Sunday, even began a short-lived run for president in 1995 on a platform that warned his fellow Republicans of the "intolerant right." He lost his job after crossing political party lines to make the toughest vote he had ever cast in his career when, in 2009, he became one of three Republicans to vote for President Barack Obama's economic stimulus bill.
Republican fury drove Specter to the Democratic Party, where he lost the 2010 primary.
Former Pennsylvania Gov. Tom Ridge, who served six terms in the U.S. House and as President George W. Bush's first Homeland Security secretary, said he thinks a serious third party could emerge on the national stage in 2016 without bipartisan agreement on major issues including the debt and immigration.
"I think the American public is fed up with the inability of both parties to find common ground," Ridge said Sunday.
Sen. Bob Casey, D-Pa., who served four years with Specter, said Sunday that he believes moderates can still bring people together.
"It's not going to happen naturally or by accident," Casey said. "We have to work at it. ... Each individual member of Congress has to take on personal responsibility. ... He has to keep the poison out of the water to avoid the kind of demonization that happens when people debate issues."
Specter, Casey said, was one of those people who could disagree without demonizing.
The other two Republicans who supported Obama's stimulus are Maine's two U.S. senators. One of them, Olympia Snowe, announced in February that she wasn't seeking re-election. She said she was frustrated by "`my way or the highway' ideologies."
In one study of congressional polarization, University of Georgia professor of political science Keith Poole mapped the political polarization of Congress by charting votes and found that the parties are more divided than at any time since Reconstruction after drifting further apart in the last 40 years.
In one essay, Poole said there are no true moderates left in the House of Representatives, and just a handful remaining in the Senate, in contrast to the Reagan era when about half of the members of Congress could be described as moderates.