The Nationals are digging in on their opposition to an Indigenous voice to parliament, although their leader has walked back from an MP’s criticism of the minister tasked with overseeing the referendum process.

“We’ve been down this road before,” David Littleproud told Nine’s Today program on Tuesday morning.

“What we fear is this will be a voice for Redfern but not for Wilcannia, not for Alice Springs and not for Carnarvon.”

The Nationals on Monday declared they will not back a proposal to enshrine a voice at a national referendum due to be held in 2023/24.

“It’s not racist to disagree with a proposal … that lacks detail and divides us on the lines of race,” Northern Territory Country Liberal Party senator Jacinta Price said.

“Minister [Linda] Burney might be able to take a private jet out to a remote community, dripping in Gucci, and tell people in the dirt what’s good for them but they are in the dark and they have been in the dark.”

Asked about Senator Price’s comments on Ms Burney, who is the Indigenous Australians Minister, Mr Littleproud said she had a lot of passion about the issue but it was important to have a respectful debate.

“It is important we bring respect to this. This should be a conversation we should be able to have,” he added.

Labor minister Bill Shorten told Nine that Ms Burney had been a great advocate for First Nations Australians.

“The issue is about whether or not we put First Nations people on the nation’s birth certificate, the Constitution,” he said.

The Labor government has called for a referendum during this term of parliament to set up an Indigenous voice — one of the recommendations from the Uluru Statement from the Heart.

Geoffrey Scott, the spokesman for the Uluru Dialogues, said the Nationals’ announcement “will only make us work harder”.

“We will continue talking with all Australians including supporters of the Nationals,” he said.

“By deciding to do this before a referendum date has even been set, or the detail has been released, it’s clear that the Nationals have put internal politics ahead of the interests of First Nations peoples.”

Ms Burney said better policies would be developed when Indigenous people were listened to.

The Nationals say the government should instead be focused on “granular” issues such as empowering local Indigenous communities and practical ways to close the wellbeing gap.

The government will release the latest Closing the Gap report on Wednesday, outlining how work on improving Indigenous wellbeing is progressing.

A Productivity Commission report released in July found five targets were not on track, including children being developmentally on track when they commence school, out-of-home care rates, adult imprisonment, deaths by suicide and sea country rights and interests.

-AAP