Ulster kept their RaboDirect PRO12 League play-off hopes alive - but only just with their fourth win in their last five games taking them to fourth in the table with six regulation games still to play.
The game ended in high drama when the Ospreys' Eli Walker snatched a late try and then nearly got in again right at the end, which would have won it for the weakened visitors and done huge damage to Ulster's play-off hopes.
However, the Welsh side left Ravenhill with a losing bonus point to their name which keeps them in second.
Ruan Pienaar's long range penalty fell short after 11 minutes but Ulster were rocked three minutes later when Pedrie Wannenburg was yellow-carded for apparently blocking a tap and go from Kahn Fotuali'i.
It was no great surprise that the Ospreys then punished Ulster by going for the corner from the penalty and even though the lineout possession was hardly clean a couple of swift rucks later saw Joe Bearman squeeze over the line. Dan Biggar converted and the visitors were 7-0 ahead after 16 minutes.
Ulster managed not to concede any further points during Wannenburg's absence and then Pienaar did make a penalty from slightly closer range just on the half hour - but then was just short three minutes later from the other side of the field.
Ian Whitten was then nearly put through after Ulster managed a midfield turnover but his inside pass was grabbed by Sam Lewis and the immediate danger passed.
But then just on half-time the Ospreys were penalised for dragging down a driving Ulster maul and two minutes later after several scrums and close-in drives referee Neil Hennessy awarded Ulster a penalty try after the referee adjudged that Biggar had deliberately prevented a try being scored by knocking the ball down after.
The Cardiff player was shown a yellow card, Pienaar converted and Ulster now led 10-7 at the turnaround and quickly made their extra man count with some great driving from their pack creating space out wide with John Afoa providing a useful pass as Paddy Wallace was put in at the left corner.
Pienaar missed the conversion but Ulster now led 15-7 after 48 minutes.
After that the game rarely came to life, although Stefan Terblanche made a crucial try-saving tackle on Hanno Dirksen just before the hour mark.
Then with two minutes to go Walker beat three Ulster tackles to make the line after a great run with referee Henessey going upstairs to the TMO before it was awarded. Biggar's conversion cut Ulster's lead to a solitary point.
Then with the final play of the game a cross-kick to Walker nearly make the line only for John Afoa to kick it dead and save the day for Ulster.