Same-sex marriage advocates have begun legal action in Australia's highest court against a controversial government plan for a postal vote on the issue, as debate continues over whether the ballot should be boycotted.

Although marriage equality is supported by a majority of Australians, the country has not joined other nations in allowing homosexual couples the right to wed amid political bickering over how the issue should be addressed.

The conservative government yesterday chose a postal vote after a proposed national ballot, which it took to elections last year, was knocked back in the upper house over opposition concerns it would spark divisive debate.

But marriage equality advocates said Canberra might be exceeding its executive authority in holding the paper ballot, with the Public Interest Advocacy Centre commencing proceedings in the High Court to stop it.

"We will be arguing that by going ahead without the authorisation of parliament, the government is acting beyond its power," said the advocacy group's chief Jonathon Hunyor.

"These are important issues about the way that power is exercised by governments and the role of parliament in our democracy."

Under the government's plans, ballots will be sent out by the Australian Bureau of Statistics from 12 September, with the outcome of the voluntary vote announced in mid-November.

If a majority of Australians vote "yes", the government would hold a free vote in parliament on the issue, with MPs not bound by party policy or by the postal ballot's result.

If there is a "no" outcome, there would be no parliamentary vote.

Supporters have called for a parliamentary vote as the best way to legalise gay marriage, with some arguing a public ballot should be boycotted as it would open the door to hate speech against the LGBTI community.

Former High Court judge and prominent gay marriage supporter Michael Kirby added he would boycott the postal vote as it was "irregular and unscientific".

"It's just a complete political improvisation and it's completely unacceptable and it should stop," Mr Kirby said.

"And I feel as a citizen I'm being treated in a second-class way."

Prime Minister Malcolm Turnbull, who supports marriage equality but has battled right-wing members of his own Liberal Party who are against legalising such unions, today spoke out against a boycott.

"I encourage every Australian to exercise their right to vote on this matter. It's an important question," he told reporters in Canberra.