Ulster's unbeaten record in the Magners League came to an end at rain-soaked Murrayfield as David Blair's drop goal put the seal on Edinburgh's win.
The visitors had led 10-8 at the break thanks to a try from centre Ian Whitten and five points from the boot of Paddy Wallace. Fraser McKenzie went over for the Gunners with Chris Paterson adding a penalty.
Ulster added two more penalties after the interval, but Netani Talei's converted try, a second penalty from Paterson and that Blair drop goal gave the hosts the four points.
Accurate kicking was hardly the order of the opening minutes, Blair putting the kick off straight out before Wallace missed a straightforward penalty and a Blair drop goal went askew.
Edinburgh kept up the pressure and after 10 minutes, Greig Laidlaw linked with Alan MacDonald to put lock McKenzie over. Paterson missed the conversion and as the rain made the ball increasingly difficult to hold.
Ulster full-back Adam D'Arcy almost unpicked the Edinburgh defence, but a try-saving tackle by Laidlaw kept the home line intact.
That gave Ulster field position for the first time and they got on the board with a Wallace penalty on the quarter-hour mark but Paterson restored the five-point margin with a 40-yard shot.
Ulster flexed their muscles in the pack as Robbie Diack thundered into the heart of the Edinburgh defence and, from the ruck, a lovely offload from lock Dan Tuohy put sent Whitten scampering over for Wallace to convert.
Ulster turned the screw prior to half-time and it took some backs-to-the-wall defending to keep the visitors out as drives by Rory Best, Tuohy and Chris Henry came to nothing.
A Paddy Wallace penalty got the scoreboard shifting after only a couple of minutes of the second half but Edinburgh hit back as number eight Talei, who scored against Northampton in the Heineken Cup last weekend, shredded the Ulster defence but held on a fraction too long with the line open for Tim Visser.
But the big Fijian finished what he had started and crashed over from close range, Paterson converted and Edinburgh started to play a more tactical game but failed to take advantage of some fine kicks from Paterson and Blair which turned the Ulster back three.
On the hour mark, Andy Kyriacou offended and Paterson slotted his second penalty but Ian Humphreys pulled the margin back to two points with a monster kick from the halfway line, giving Ulster a real boost going into the final minutes.
It was nerve jangling stuff until Blair showed his cool and slotted a long-range drop goal which deservedly clinched the points for Edinburgh.