A new study co-authored by a High Court judge has said a controversial paper published last year claiming Irish judges used Wikipedia as a source for their rulings, is flawed and should be retracted, or required to undergo major changes.
The new study says citations in judgments are driven mainly by submissions and claims there is no "Wikipedia effect".
The authors of the original paper say they disagree with the criticisms made in the new study, but they also say they welcome a discussion in the context of additional data gathered as part of the latest research.
The study published today, co-authored by Mr Justice Richard Humphreys along with six former or current judicial assistants, claims the methodology used in last year's paper was flawed.
Mr Justice Humphreys said there were "fundamental problems" with the original paper, which rendered its conclusions unreliable. The judge said the "baseless speculations" seemed to come from a lack of knowledge about how judgments are produced, and he did not believe there was bad faith by the authors.
In the original paper, researchers at Massachusetts Institute of Technology (MIT) and Maynooth University arranged for 154 articles on Irish Supreme Court decisions to be written by law students. Half were uploaded to Wikipedia and the rest were kept offline as a control group.
They discovered the cases mentioned in the uploaded articles were 20% more likely to be cited as precedents in court rulings than similar cases which had not been uploaded.
They also claimed an analysis of the language used in the judgments found the "linguistic fingerprints" of the Wikipedia articles.
'Headline grabbing claims'
The new research carried out by Mr Justice Humphreys along with three High Court judicial assistants, one judicial assistant from the Court of Appeal and two former High Court judicial assistants is published in the Irish Law Times today and is strongly critical of the previous paper's "headline grabbing claims".
It claims the original authors worked off a commercial website rather than the official courts.ie website where judgments are published, leading to the possibility of consistency issues. And it claims the authors compared too long a period of citations before the articles appeared on Wikipedia with a short period of around two years afterwards.
In the new research, using different methodology, the authors say their comparison showed citations of the cases mentioned on Wikipedia and those in the control group rose by the same amount, meaning, they say, there was no effect whatsoever from the cases being mentioned on Wikipedia.
The authors also say those involved in the original study had no direct evidence of even a single case of a judge searching Wikipedia to find a citation or a single piece of judicial prose remotely comparable to the efforts of an undergraduate on a Wikipedia page.
They say judicial outputs are public and up for scrutiny, but they continue that it is important in the public interest that public respect for institutions is not undermined by what they describe as "flawed science".
Mr Justice Humphreys and his co-authors said their research found the large majority of citations in judgments are driven by references in written or oral submissions. And a small number of other references can be explained, they say, by reference to verifiable prior judicial knowledge. They say judicial prose bears no demonstrable relationship to Wikipedia and they say, the claims of the original authors do not stand up to scrutiny.
In response, one of the original authors, Dr Brian Flanagan, an associate professor at the school of law and criminology at Maynooth University said their analysis stood up to the criticism levelled at it. He said their use of commercial data was cross-checked against the court’s own records and only small differences were found which did not change their conclusions.
Dr Flanagan said it was plain to see from their research that cases which had been added to Wikipedia jumped immediately in terms of citations by judges, after being added - a jump he said was notably different to any period in the two years beforehand.
However, Dr Flanagan said the original authors understood the authors of the new research had gathered additional data from judgments and they welcomed that discussion.
The High Court President, David Barniville welcomed the publication of the new study. He said those with practical knowledge of judicial decision making knew the claim that judges relied on Wikipedia in any material way was "plainly wrong" and the new study confirmed such claims were wholly inaccurate.
High Court judge, Mr Justice Cian Ferriter said the new findings confirmed that case law cited in Irish court judgments came from parties’ written or oral submissions and not from Wikipedia.
