His resignation comes a day after he was named NATO's next secretary general.
Mr Rasmussen, leader of a centre-right coalition since 2001, was named yesterday as the next head of the transatlantic military alliance.
He will succeed Jaap de Hoop Scheffer, who steps down at the end of July.
Mr Rasmussen arrived at the royal palace at around 12.25pm (Irish time).
He left less than half an hour later, making no comment to the crowds or journalists gathered outside.
Finance Minister Lars Loekke Rasmussen - no relation - is expected to take over as prime minister.
Under Denmark's electoral system, in which parties rather than individual politicians are elected to parliament, the constitution does not require new elections to be called if the prime minister resigns.
