"It is time for a change -- it is time for Main Street to have a champion at the Fed," Boxer said in a statement. "Dr. Bernanke played a lead role in crafting the Bush administration's economic policies, which led to the current economic crisis. Our next Federal Reserve Chairman must represent a clean break from the failed policies of the past."
It's not clear whether Bernanke's confirmation is in jeopardy, because he is likely to garner some Republican support. In the Senate Banking Committee, four Republicans voted to confirm Bernanke, crediting him for saving the economy from a second Great Depression.
Senate Republicans were also trying to figure out on Friday who would be supporting and opposing Bernanke's confirmation
But the Senate can't even start the process of considering Bernanke until 60 senators sign off, because a few senators who oppose his confirmation filed official "holds" delaying the process.
0:00/4:56Ben Bernanke's biggest year"The math to 60 -- at this point -- looks bad for Bernanke," wrote Chris Krueger, an analyst for Concept Capital Washington Research Group in a report. "Chaos is reigning on the Hill right now and Democratic members are in severe anxiety over their own re-elections."
Some are really starting to wonder whether Bernanke will be confirmed before Feb. 1. If the vote is delayed, there's a question as to whether Bernanke can be temporarily re-appointed as acting chair. If not, Fed Vice Chair Donald Kohn would serve as acting chairman.
Senate leaders were unsure Friday how the votes were going to play out or when they'd start the procedure to force a vote.
Even Senate Majority Leader Harry Reid issued a statement late Friday saying he'd support Bernanke, but "my support is not unconditional."
The White House is also getting involved, although spokesman Bill Burton declined to "engage in hypotheticals" about whether the president believed Bernanke's confirmation was in trouble.
"He continues to think that he's the best person for the job, and will be confirmed by the United States Senate," Burton said to reporters earlier Friday on Air Force One. "But of course people on the White House staff and around the President continue to work to get that done."
Bernanke has always had his critics in the Senate. Bernie Sanders, a left-leaning independent from Vermont who often votes with the Democrats, and Jim Bunning of Kentucky, Sanders' political opposite, are two of the most vocal.
Up until Tuesday, insiders believed that Bernanke had locked up more than 60 votes necessary to break a Senate filibuster. In December, he won solid support from the Senate Banking committee.
However, the political landscape changed when Republican Scott Brown won the special election to fill the Massachusetts vacancy in the Senate, once considered a Democratic stronghold.
Sen. Sherrod Brown, D-Ohio, voted for Bernanke in committee but told CNN on Friday that he hasn't made up his mind about his final vote on the floor. Brown said he doesn't think a delay in the confirmation vote will impact market stability.
"The markets understand that there is dissension and contentiousness about Chairman Bernanke," Brown said.
* CNN's Dana Bash and Ed Henry contributed to this report.
Senate panel approves Bernanke for second termHarry Reid helps to build up support for Bernanke
