Schofield and Willoughby, 42, previously presented the ITV skating competition together for years, but he stepped down from presenting it last May along with This Morning, which they also co-hosted, after he admitted to having an "unwise, but not illegal" affair with a younger male colleague.
Willoughby herself quit This Morning after 14 years after learning of an alleged plot to kidnap and murder her.
She finally returned to screens on Sunday following a three-month break with Stephen Mulhern, 46, replacing Schofield as her co-host.
According to The Sun, a source said: "Phil is still bruised by the whole Holly falling-out and the way it was played out publicly. He still can’t bring himself to watch This Morning and even listening to the theme tune is deeply triggering.
"It would seem unlikely he is avidly tuning in to watch Holly’s return to TV. Phil just wants to move on and live a quiet life away from the spotlight whilst he tries to rebuild.
"He reached out to offer his support when the awful kidnap news broke but it’s understood they have not been in contact since, so he’s probably not spoken to Holly ahead of her DOI return."
Mulhern, who previously filled in for Schofield on the show when he got struck down with Covid, proved a hit with viewers on the launch show.
One talking point that had people online in stiches, was when boxer and contestant Ricky Hatton jokingly "punched" Mulhern, resulting in him ending up on the floor.