A jury which convicted Robert Xie of bludgeoning five relatives to death was provided with "misleading, confusing and unfairly prejudicial" DNA evidence, his appeal has been told.
This included expert testimony about a stain found on Xie's garage floor - 300 metres from the murder scene - 10 months after the killings, his barrister Belinda Rigg SC said on Monday.
After four trials, a jury in 2017 found Xie guilty of murdering three adults and two children in the bedrooms of their Sydney home in the early hours of July 18, 2009.
Xie's newsagent brother-in-law Min Lin, 45, his wife Lily Lin, 43, the couple's sons Henry, 12, and nine-year-old Terry, and Lily's 39-year-old sister, Irene, suffered horrific head injuries when they were attacked with a hammer-like object attached to a rope.
The Lin's daughter Brenda was on an overseas school excursion at the time and went to live with her uncle and his wife Kathy after the murders, before his arrest.
The now 56-year-old Xie is behind bars for the rest of his life and appeared via video-link from Lithgow jail at his Court of Criminal Appeal hearing.
Ms Rigg said three of his eight appeal grounds related to DNA evidence, including a claim that evidence given by scientist Dr Mark Perlin led to "a miscarriage of justice".
At the Supreme Court trial, the Crown contended the garage stain, item 550, was blood which "most likely" contained DNA of four Lin family victims and which Xie "transferred" immediately after the murders.
Dr Perlin's "quantitative probabilistic analysis" of the item was an important plank of the Crown's circumstantial case, Ms Rigg said.
Dr Perlin was involved in the development of a computer-based program called TrueAllele which he used to give complex statistical DNA evidence about item 550.
He testified that Brenda Lim's DNA "does not statistically support it to be present in that mixture".
"However the applicant contends that the aspects of Dr Perlin's evidence which would have been of importance to the jury were not wholly or substantially based on specialised knowledge, were misleading and confusing, and were unfairly prejudicial," Ms Rigg said.
Ten blood relatives, including the five people killed, had been at Xie's home including his parents, his wife Kathy and niece Brenda, the barrister added.
At trial, the defence contended the stain wasn't shown to be blood, while the evidence could not exclude the possibility that DNA from other relatives including Brenda was in the mix.
This would rebut any suggestion the DNA related to the murders as Brenda was still alive.
In written submissions before the three Appeal Court judges, the defence said "the process of explaining the problems with Dr Perlin's evidence is a gruelling and tedious one".
The issues could not be addressed "very quickly and easily", but unless courts were to simply defer to experts when the material is not readily understandable, it needed to be undertaken.
The hearing has been listed for seven days.