AI Predicts Judicial Outcomes
October 24, 2016
judicial decisions of the European Court of Human Rights (ECtHR) have
been predicted to 79% accuracy using an artificial intelligence (AI)
method developed by researchers at UCL, the University of Sheffield and
the University of Pennsylvania.
The method is the first to predict
the outcomes of a major international court by automatically analysing
case text using a machine learning algorithm. The study behind it was
published today in PeerJ Computer Science.
“We don’t see AI replacing judges or lawyers, but we think they’d find
it useful for rapidly identifying patterns in cases that lead to certain
outcomes. It could also be a valuable tool for highlighting which cases
are most likely to be violations of the European Convention on Human
Rights,” explained Dr Nikolaos Aletras, who led the study at UCL
In developing the method, the team found that judgements by the ECtHR
are highly correlated to non-legal facts rather than directly legal
arguments, suggesting that judges of the Court are, in the jargon of
legal theory, ‘realists’ rather than ‘formalists’. This supports
findings from previous studies of the decision-making processes of other
high level courts, including the US Supreme Court.
“The study, which is the first of its kind, corroborates the findings of
other empirical work on the determinants of reasoning performed by high
level courts. It should be further pursued and refined, through the
systematic examination of more data,” explained co-author Dr Dimitrios
Tsarapatsanis, a Lecturer in Law at the University of Sheffield.
The team of computer and legal scientists from the UK, alongside Dr
Daniel Preoţiuc-Pietro from the University of Pennsylvania, extracted
case information published by the ECtHR in their publically accessible
“Ideally, we’d test and refine our algorithm using the applications made
to the court rather than the published judgements, but without access to
that data we rely on the court-published summaries of these
submissions,” explained co-author, Dr Vasileios Lampos, UCL Computer
They identified English language data sets for 584 cases relating to
Articles 3, 6 and 8* of the Convention and applied an AI algorithm to
find patterns in the text. To prevent bias and mislearning, they
selected an equal number of violation and non-violation cases.
The most reliable factors for predicting the court’s decision were found
to be the language used as well as the topics and circumstances
mentioned in the case text. The ‘circumstances’ section of the text
includes information about the factual background to the case. By
combining the information extracted from the abstract ‘topics’ that the
cases cover and ‘circumstances’ across data for all three articles, an
accuracy of 79% was achieved.
studies have predicted outcomes based on the nature of the crime, or the
policy position of each judge, so this is the first time judgements have
been predicted using analysis of text prepared by the court. We expect
this sort of tool would improve efficiencies of high level, in demand
courts, but to become a reality, we need to test it against more
articles and the case data submitted to the court,” added Dr Lampos.
*Article 3 prohibits torture and inhuman and degrading treatment (250
cases); Article 6 protects the right to a fair trial (80 cases) and
Article 8 provides a right to respect for one’s “private and family
life, his home and his correspondence” (254 cases).