Eight days after upending champions Leinster in the RDS, Ulster slumped to defeat in Swansea, with Ospreys fly-half Stephen Myler kicking 14 points to steer the hosts to victory.
The home side had to rely on an heroic defence, led by back-rowers Morgan Morris and Jac Morgan, to restrict their opponents to one penalty try, despite the visitors dominating in terms of possession and territory.
Morris scored the Ospreys' only try, with Myler kicking four penalties and a conversion.
Ulster responded with their penalty try and John Cooney added two penalties.
FULL MATCH DETAILS
Dan Evans was a late withdrawal for the Ospreys with a hip injury, before the home side came firmly under the cosh in the opening stages.
The Ospreys, who beat Munster in their last home fixture, conceded six penalties in the opening 13 minutes as Ulster battered the home defence, but somehow the hosts kept their line intact.
It therefore took a penalty try award for Ulster to open their account, with Luke Morgan being adjudged to have deliberately knocked on to prevent a try.
Morgan was yellow-carded before Myler and Cooney exchanged penalties to leave the Ospreys trailing 10-3 at the end of the first quarter.
Morgan returned from the sin bin before the Ospreys suffered a further injury blow when Evans' replacement, Max Nagy, left the field with a shoulder injury to be replaced by Gareth Anscombe.
The second quarter was vastly different to the first, with the Ospreys benefiting from a number of penalties in their favour to have an equal share of the contest, and they drew level when Morris stormed over from close range.
Myler converted and kicked a penalty as Ulster lost Welsh international hooker Bradley Roberts to a leg injury, leaving the visitors to wonder how they trailed 13-10 at the interval.
Five minutes after the restart, Ulster brought on international prop Jack McGrath for his first appearance in over a year as Dan McFarland's side resumed their domination.
Cooney and Myler exchanged penalties as the second half mirrored the first and the Ospreys grew in influence as the half progressed.
Myler kicked a fourth penalty after the visitors' scrum had infringed and the home side went on pick up a valuable four points, which for large parts of the match looked highly unlikely.