UK public borrowing fell in May from the sky-high levels of a year earlier when the government was ramping up a huge spending response to the pandemic.
There are also signs that the recent recovery in the economy is boosting tax revenues.
Borrowing excluding public sector banks totalled £24.3 billion in May, down from £43.8 billion the same month in 2020, the Office for National Statistics said.
Economists polled by Reuters had on average expected a shortfall of £26 billion.
The May figure took the deficit for the first two months of the 2021/22 financial year to £53.4 billion.
That was the second-highest on record but was down by almost £38 billion from the April-May period of last year.
Borrowing is falling faster than the UK government's budget forecasters had expected.
But Samuel Tombs, with consultancy Pantheon Macroeconomics, said Britain's longer-term economic problems would continue to weigh on the public finances.
"We still think that the government will have to stick to plans to hike corporation tax in 2023 and to increase the effective income tax rate by freezing existing thresholds, if it wants to ensure that public borrowing declines below 4% of GDP in the mid-2020s," Tombs said in a note.
He said the figures for April and May had been helped by a delayed recording of a payment to the European Union.
Revenues flowing into the public coffers in May rose by £7.5 billion from a year earlier to just under £57 billion, reflecting the pick-up in the economy as coronavirus restrictions were lifted.
UK government spending in May fell by almost £11 billion to just under £82 billion.
Finance minister Rishi Sunak has committed to measures worth £350 billion since the start of the pandemic last year, one of the biggest Covid-19 support programmes among the world's major economies and saddling the public finances with a deficit equivalent to 14.3% of gross domestic product in 2020/21.
Sunak, responding to today's data, repeated his pledge to "get the public finances on a sustainable footing" and said he had taken "difficult but necessary steps" in his budget plan announced in March.
The spending surge that Sunak authorised has begun to slow as the number of workers on his job-protecting wage subsidy scheme falls. The furlough scheme is due to be scaled back from July before ending altogether in September.
Sunak has resisted pressure to extend most other support measures despite Prime Minister Boris Johnson's decision to delay the lifting of the last social-distancing rules which had been due to take place on Monday.
Reports have suggested that Sunak was considering blocking an automatic increase of nearly 6% in old-age pension payments as part of a wider effort to rein in costs.
The Office for National Statistics said Britain's public debt mountain reached almost £2.2 trillion in May, or 99.2% of GDP, the highest since the 1960s.