Anti-racism campaigners will gather in Federation Square to protest against the far right-wing United Patriots Front.
Members of United Patriots Front unveiled the "Stop the mosques" banner at the Collingwood AFL game on Friday night.
It sparked strong condemnation from the AFL, the club and political figures, including Foreign Minister Julie Bishop.
"They're not AFL people - they're representing extreme minority views.
"They're not welcome at our game and they're not supporters of our game," AFL chief executive Gillon McLachlan told reporters on Saturday.
The anti-racism protest will begin at 2pm.
On Saturday Blair Cottrell from the United Patriots Front released a video via social media 'explaining the reasoning' behind the sign.
The sign was displayed by the scoreboard at the City End of the ground at halftime during the AFL clash between the Magpies and Richmond.
Cottrell claimed the group's aim was not to "stir up trouble" but rather to highlight their "concern" about the future of their country.
"Call it a hunch, but I think if this country stopped building places of worship and segregated communities within communities for a foreign power, which doesn’t like us very much, I think our country might have a brighter future," he said in the video.
"You think I’m pedalling fear? I’m concerned with the future of my country and I’m realistic about the people that are being brought into this country - that they aren’t like us and never will be like us.
"They’re a foreign people, a foreign culture with foreign traditions that go back thousands of years. They’re nothing like us. That’s the truth. That’s reality."
Collingwood is looking to ban those responsible.
A video of the banner being put in place was later uploaded to the Facebook page of far-right group the United Patriots Front.
The AFL condemned the incident in a statement following the game, saying "such actions have no place in society, and not in our game".
"Match day security removed the banner when they became aware of it and evicted the patrons responsible," the statement said.
— Collingwood FC (@CollingwoodFC) April 1, 2016
The AFL will follow up the incident with MCG security and Victoria Police.
The banner, which also said "Go Pies", drew swift condemnation from Collingwood, with president Eddie McGuire promising bans if the culprits are found to have any official connection to the club.
"The club awaits the outcome of further investigations into the matter by the AFL, the Victoria Police and MCG security," a club statement read.
"Regardless, these people do not speak for Collingwood and are condemned by Collingwood. If it established that they have a formal connection to the club, this connection will be severed.
"There is no place at Collingwood, or in our game, for such behaviour."
According to the AFL website Collingwood president Eddie McGuire said the club would help police with an investigation.
"If they have anything to do with our club, they'll be banned," he told Fairfax.
"Get these people and make an example of them, they should be banned for life."