Malcolm Turnbull will ask the states to ban parole for violent offenders when he meets with the premiers on Friday.
Gunman Yacqub Khayre was on parole when he shot dead a clerk in a Melbourne apartment block on Monday night, and was later killed in a shootout with police.
Khayre had served jail time over a violent burglary in 2012 and had been on parole for arson since November.
He spent 16 months on remand before being acquitted of the 2009 Holsworthy army barracks terror plot in Sydney.
The prime minister, who will convene the Council of Australian Governments in Hobart on Friday, said the issue of parole was a high priority.
"There have been too many cases of people on parole committing violent offences of this kind," Mr Turnbull told reporters in Canberra on Tuesday.
He questioned why Khayre was released on parole, given his extensive criminal history and links with terror.
"I have raised these today with the Victorian premier, whom I called last night and I called again this morning," Mr Turnbull said.
"How was this man on parole? He had a long record of violence. A very long record of violence. He had been charged with a terrorist offence some years ago and had been acquitted."
The issue was clear-cut for Justice Minister Michael Keenan.
"Violent offenders should not be on parole and the prime minister will be raising this important issue with premiers on Friday," Mr Keenan told AAP.
Opposition Leader Bill Shorten said it was baffling that the man was on parole.
"If we know the person is a criminal, what are they doing out on the streets?" Mr Shorten told reporters in Brisbane.
"I'm sure the Victorian government is going to answer it."
Victorian Premier Daniel Andrews said the gunman had not only been eligible for parole and received it, but had complied with all its terms and conditions, including drug testing and a curfew.
"We'll look at that very closely, of course, and if there are any changes in any element of this act - which we're considering as an act of terror - then we stand ready to make them," Mr Andrews said of changes to parole.
He noted Khayre had been acquitted of the NSW charges.
Parole in Victoria was overhauled after the 2012 murder of Jill Meagher, leading to a drop in parolees committing serious crimes from 60 in 2013/14 to 13 in 2015/16.
A study by the NSW Bureau of Crime Statistics published in 2014 found seven per cent of violent offenders on parole committed a further violence offence, and the likelihood of reoffending significantly dropped where the quality and intensity of supervision is improved.
Hanson calls for 'Trump-like' Muslim ban
Pauline Hanson has called on the prime minister to look seriously into temporarily banning Muslims from migrating to Australia - or, at the very least, more stringent vetting of those wanting to come to the country.
The One Nation leader has written to Malcolm Turnbull, compelled by the recent terrorist attacks in the UK, Europe and Melbourne.
Successive governments had failed to do what was needed to stop the growth and spread of Islamic terrorism, she said.
"If laws need to be changed then so be it; if you need to consider a US style ban, as proposed by United States President Donald Trump, then so be it," Senator Hanson wrote.
She also wants security forces to be given greater powers to deal with people on terror watch lists before they become violent.
On top of that, she suggests changing the way the Administrative Appeals Tribunal - an independent review body - works and its ability to overturn decisions regarding immigrants and refugees.
"If we continue to turn a blind eye to the radicalisation that is occurring in mosques throughout Australia then, I fear, we will soon see terrorist attacks to the degree that is happening in the United Kingdom and elsewhere around the world," Senator Hanson said.