Steven Davis scored with the last kick of the ball to give Rangers a Champions League lifeline against Malmo.
A two-minute lapse in Sweden could yet prove costly but a deflected strike from Davis in the fifth minute of stoppage time reduced the deficit to 2-1 midway through the third qualifying round tie.
Rangers goalkeeper Allan McGregor only needed to make one first-half save but he was beaten twice inside the opening four minutes of the second period.
Goals from Soren Rieks and Veljko Birmancevic put Jon Dahl Tomasson's side in the driving seat for a place in the play-offs, where Olympiacos or Ludogorets will be waiting.
But Davis - who played for Rangers in a Champions League qualifying defeat by Malmo 10 years ago - struck a powerful left-footed effort from 20 yards which hit a defender and wrongfooted the home goalkeeper.
Steven Gerrard made seven changes for Rangers' first European game of the season with Cedric Itten starting amid the continued absence of Alfredo Morelos. As well as the suspended Leon Balogun, Glen Kamara and Kemar Roofe, Gerrard was also missing Joe Aribo.
Malmo started with two up front and Rangers took time to settle but McGregor's only first-half concern was a low effort from Jo Inge Berget, which he dived to save.
Malmo were leaving space at the back and Rangers had first-half chances. Scott Arfield had an effort saved and Ryan Kent and Scott Wright both caused problems without testing the goalkeeper.
The game turned just after the interval.
The Rangers defence got dragged to the left and Birmancevic crossed for Rieks to volley home in space.
Tavernier soon misplaced a pass near the halfway line and the backtracking Connor Goldson could not cut out the pass, leaving Birmancevic to net.
Rangers took time to respond but they pushed Malmo back for the final 20 minutes and Davis struck in the nick of time.
Manager reaction
After the game, Gerrard hailed Davis' impact.
"I'm disappointed with the result and the goals we conceded but that was a big moment at the very end of the game, and it sets it up for next week," Gerrard said.
"It's not the position we wanted to be in but it's obviously a lot better than going in at 2-0 and we were also on the verge of maybe going in even worse as well.
"The injury time has been very big for us but our performance over 90 minutes hasn't been enough. So we are going to have to go up levels individually and collectively next week.
"I'm going to call on the Ibrox crowd. I've never done it before but we are going to need them next week, for a full 90 minutes, in full voice. And with that behind us, and a mentality change, then that injury time could be a big moment."
Gerrard added: "We had two magnificent performances. Fil Helander I thought played at Champions League level, he was outstanding, and Steven Davis.
"I'm running out of plaudits for Davo but he's someone who doesn't give up, keeps going to the end and he has provided a real big moment that's given us an important lifeline."
Gerrard, who hopes to have Joe Aribo back for Saturday's trip to face Dundee United after the midfielder missed out in Malmo, added: "I don't think we deserved to lose the game 2-0 because at half-time we were in a good shape and looking more likely but we lost our way for four-five minutes and lost two goals.
"We never really cleared our heads after the first goal and were punished again.
"I've just said to the boys in there, my honest opinion is there's six or seven who need to raise their level and raise it sharply."
