NEW YORK -- This one had all the earmarks of another ugly collapse.

The Orioles found themselves down 3-0 in the first inning on Sunday, with their ace on the mound and coming off a depressing defeat on Saturday after allowing seven runs in the final two innings.

Then, almost as quickly, the Orioles rebounded -- something they haven't done much of over the past several seasons -- to beat the Yankees, 6-4, at Yankee Stadium to win the three-game series.

"That was a big win, it was," said designated hitter Kevin Millar, who hit the first of a pair of decisive home runs on Sunday. "Give up a three-run home run with two outs in the eighth and then a grand slam to walk it off [as the Orioles did on Saturday], teams tend to get down. We came back in here loosey-goosey, playing the music, having a good time and believing in ourselves."

The game opened with the Yankees taking a 3-0 lead in the first off Erik Bedard, courtesy of a run-scoring sacrifice fly by Bobby Abreu and a two-run home run by Saturday's hero, Alex Rodriguez.

That made 10 runs allowed in their last three innings against a team the Orioles haven't had a .500 record against in 10 years in a place they haven't won as many as four games in almost 20 seasons.

But after their comeback on Sunday -- which included a save by Chris Ray, who gave up the game-winning hit on Saturday -- the Orioles are 2-1 against the Yankees this season in The House That Ruth Built in just the first week of the season.

That comeback was keyed by Bedard, who, after allowing A-Rod's home run, retired 11 in a row and 20 of the last 22 batters he faced. The left-hander finished with three runs allowed on five hits over seven innings, striking out five.

"He didn't let the runs in the first bother him," pitching coach Leo Mazzone said. "He pitched, he changed speeds, he threw his curveball and his changeup and looked very confident on the mound. He looked like a classic left-handed pitcher today."

As do the Orioles, Bedard has had a history of letting earlier adversity keep him from having success. In his first start of the season, in Minnesota, Bedard gave up 10 hits and six runs in 4 2/3 innings. It looks as though the 28-year-old may have turned the corner.

"I guess with more experience under my belt I have a better way of dealing with those situations," said Bedard, who credited throwing his curveball better for strikes this time out.

Millar started the rally with a two-run home run against Yankees starter Darrell Rasner in the second, getting the Orioles to within one.

But the big blow came when backup catcher Paul Bako belted a three-run blast off Rasner in the fourth, putting the Orioles up, 5-3.

Sharing starting duties while Ramon Hernandez sits on the disabled list, Bako hit his first homer since Aug. 21, 2004, when he was with the Cubs.

"[The ribbing] was not really as bad as I thought it was going to be," said Bako of his teammates' reaction to the rare power stroke. "They took it pretty easy on me. [First baseman Aubrey] Huff said that [outfielder Nick] Markakis called it. When it worked out, Huffy told him that he should play the lottery. That was some fun."

The Orioles could have had more runs and fun, as they loaded the bases in the fifth against Rasner and reliever Sean Henn with two out. But outfielder Corey Patterson was robbed of a hit when his liner along the first-base line was snared by a diving Josh Phelps.

Bedard (1-1) and the bullpen took care of the rest.

The Yankees threatened in the eighth by loading the bases with two out, but left-hander John Parrish -- making his fifth appearance of the season -- got Phelps to fly out.

Ray, touched for the A-Rod grand slam on Saturday, came on to pitch the ninth with barely a memory of that game. Yet a few other Orioles had that game in mind when Ray entered.

"We were thinking about that after we got him in there," manager Sam Perlozzo said. "I think you saw a different Chris Ray today. He settled down, he took his time, used all his pitches and he was effective."

Ray spent much of Saturday night reflecting on that day's loss. Maybe he wasn't obsessing, but the home run was definitely dominating his thoughts.

"When I first got here this morning, I wanted another shot at it. I wanted to go back out there as soon as possible in a save situation," said Ray, who came in with a two-run lead and allowed just a Johnny Damon single with one out to earn his second save of the season. "This is the best-case scenario; we still take the series, and I was able to go out there and get a save."