Australia’s big four banks could be in line for a massive hit to their balance sheets thanks to a new levy to be unveiled in Tuesday’s federal budget.

Banking sector sources told The Australian Financial Review that Treasury Secretary John Fraser would contact banking chiefs at 6.30pm, before Treasurer Scott Morrison was due to deliver his budget speech.

The tax will reportedly raise around $6 billion over four years.

Responding to the budget speculation, Australian Bankers’ Association chief executive Anna Bligh said banks were already significant contributors to the tax bucket in Australia.

“I think they would be very concerned if there is a punitive tax or levy that singles banking out from the rest of the business sector,” she told ABC radio.

“There would be a great deal of alarm not only in the banking sector but in both the domestic and international investment communities if there was to be an attack on the value of banking in this country.”

ABC reported that a raft of sluggish bank results and news that the levy was likely to be unveiled in the budget had fuelled a big sell-off of bank stocks.

CBA shares dipped 2.68 per cent to $83.01, Westpac was down 2.63 per cent to $33.18, ANZ was down 2.2 per cent to $29.29 and NAB fell 1.03 per cent to $32.76.

Fairfax reported that the bank stocks shed a total of $13 billion on Tuesday.

Ms Bligh, a former Queensland Labor premier, said such a levy — touted as a way to help fund a compensation scheme for victims of misconduct — could be a slippery slope.

Westpac chief Brian Hartzer warned it would set a precedent to ask the banks to cover remediation for past issues out of their control.

“We’re supportive of people carrying their weight, but we also note a lot of the big issues have actually been things outside the banking sector,” he said.

Australian Chamber of Commerce and Industry chief James Pearson urged caution in imposing new levies on any sector.

“Clearly there have been for some time now community concerns about some aspects of bank practices and behaviour but equally I think most observers would recognise that the banking industry has done a lot to address those concerns,” he told Sky News.

Mr Pearson expected if the market can stand it, the banks will find a way — as any company would — to pass on as much of their costs to customers.

But Australian Small Business Ombudsman Kate Carnell said if a levy to pay for a compensation scheme was true it would be a good way forward.

The Turnbull government has already asked the Productivity Commission to hold an inquiry aimed at ensuring Australia’s financial system is competitive and innovative.

It will form part of the government’s response to the Financial System Inquiry that was conducted by David Murray in 2014.

The inquiry will start on July 1 and is due to report to the government mid-2018.