Introduction
Every legal proceeding in a courtroom is bound to get scrutinized and get associated with the outcome predictions either by the spectators or the advocates involved in the proceedings. Outcome prediction is intertwined with a successful legal practice. These predictions by lawyers (advocates in particular) are the result of their years of practice, knowledge of law, court craft and the facts of the particular case(s).
However, the Supreme Court’s compass on outcome predictions seems to have deflected. The Hon’ble Supreme Court in its recent order in the case of Madhavendra L. Bhatnagar v. Bhavna Lall (hereinafter “Bhatnagar v. Lall”)[i]has opined that prejudging or speculating the outcome of a case by an attorney is “bordering on professional misconduct“. Finding this ‘disconcerting‘, the Court initiated suo motu action against the concerned advocate. The stance of the Court ignores that we live in the era of data science, where sophisticated technologies, specifically predictive analytics allow for accurate outcome predictions. In this light, predictive justice today is associated with analysing the available judgments and applying them to the new cases to predict their likely outcome using algorithms.
The difficulty in juxtaposing Supreme Court’s opinion with the developments in predictive analytics has been the motivation for this article. It is an attempt to dwell into the significance of predictions in the practice of law. Conventionally it was done by lawyers using traditional methods, but the precision offered by new technological tools has caused a paradigm shift in the approach. With these substantial changes, outcome prediction has been redefined and lately has gained significant prominence in the field of computational legal research with more and more scholars and policy makers looking for opportunities for its application in actual process of justice administration.
The article briefly discusses theoretical basis of prediction in lawyering followed by analysing the Court’s opinions given in the aforementioned case in the context of various research studies on this topic. Lastly, the authors shall examine the advancement in the data analytics tools and the benefits offered by them.
Prediction vis-a vis Legal Reasoning
“Predicting court decisions has always been the goal of every lawyer and every academic consultant. Whoever turns to one or the other of these actors expect from them, with more or less hope, a foreknowledge of jurisprudence.”[ii]
The wide prevalence of outcome prediction among lawyers makes it tremendously intuitive for them. The ubiquitous nature of this skill explains the ready acceptance of the prediction theory when it was articulated by Oliver Wendell Holmes Jr. in his speech in Boston in 1897 to an audience of lawyers and judges. The prediction theory given by Holmes became so popular that it led to the inception of a new school of thought in jurisprudence called legal realism.[iii]
Holmes’ notion differed from strict deductive legal reasoning which never fully captured the mechanism of how a judge arrives at a decision.[iv]This deductive reasoning essentially involves arriving at a decision based on a set of legal premises. The political, moral or policy considerations could have no appeal to a judge in his/her decision.[v]However Holmes believed that judicial decision making is a result of the play among multiple forces. The spectrum of these forces covers historical, social, psychological, political and economic dimensions. This incorporation acknowledged a skill set, already being deployed by advocates in their litigation strategies.
The Art or Science of Predicting
In the case of Bhatnagar v. Lall, the Supreme Court directed the party to file an affidavit and reveal the name of an advocate who predicted the outcome of a pending case before the Court. As per the facts of the case, the husband filed an application for grant of an interim injunction against his wife to restrict her from bringing an action against him in Arizona. When it reached the Apex Court, the Court reprimanded the advocate who had suggested that initiating proceedings in Indian courts would be futile.[vi]
It must be noted that a lawyer assumes multiple responsibilities, one of which is of an advisor. This role necessarily involves predicting outcome and, on that basis, advising the clients the most prudent course of action.[vii] Prediction is a sine qua non for taking an informed decision both for the lawyers as well as the clients. In essence, it involves weighing the feasibility of all the approaches that can be taken. However, the Supreme Court expressed its failure to understand how an advocate is able to “prejudge the outcome of the proceedings or if we may say so speculate about the outcome thereof”. [viii]
Such an opinion at a time when advancement in analytics is allowing predictive justice suggests that the judicial opinion is not in tandem with the progress in the field of legal technology. Consider Figure 1 given below, which shows the level of accuracy achieved by artificial intelligence (AI) in predicting outcomes.
The level of accuracy by algorithm surpassed lawyers working in the same field. For instance, only 59.1% of expert lawyers predicted correct outcome in the study by Ruger et al. (2004)[ix]in comparison to 75% accuracy by predictive algorithms. Similarly, UK based legal tech start up, CaseCrunch showcased 86.6% accuracy by their algorithms, while the lawyers had an overall accuracy of 62%.[x]
Research | Accuracy | Subject Matter |
86%-92% | Criminal law cases related to murder (India) | |
70.2% | US Supreme Court cases | |
79% | Decisions by European Court of Human Rights (ECtHR) | |
75% | US Supreme Court cases |
Figure 1: Accuracy achieved in outcome prediction in various studies.
Further, the predictive analytics work better in common law traditions as the decisions have to work as continuation with the thread of stare decisis. This explains why the aforementioned studies were conducted in common law countries and thereby able to achieve such high accuracy.
One can, therefore, hope that the order of the Supreme Court is just an aberration as there is no significant case where the Court has dealt with this matter. However, prediction is being done by the courts whenever they make a decision as it is geared towards its likely impact in the future.
Moving Towards Predictive Justice
Legal-tech industry has various tools to offer that can predict cases by collecting and analysing vast amount of data. ‘Premonition’ is one such tool which claims to predict the success rate, case duration and pairing with the judge of a lawyer. Similarly, ‘LexMachina’ allows users to see how a judge has ruled on cases in the past and thereby indicating the whether the case can have a favourable outcome or not. Other tools like ‘Ravel Law’, ‘Intraspexion’ etc., work in a similar manner.[xi] One common feature among all these is the requirement of building large databases. The larger the database, the less biased and more accurate it will be.
These tools have immense benefits in the legal practice. It can encourage parties to approach out of court settlement or alternate dispute resolution systems in cases where the likelihood of success is low. It can also help in cases where violation of law seems likely, thereby prioritise cases where judicial expertise is needed as opposed to cases requiring simple application of law. This can be advantageous in India as well where the pendency of cases is high. It would result in efficient allocation of judicial resources.
Predictive models also have the capacity to exclude legally irrelevant factors in their predictions thereby cancelling out arbitrary factors in judicial decision making. This could check misuse of judicial discretion and bring more consistency in application of law.[xii]
However, lack of adequate understanding of how algorithms arrive at a particular conclusion causes opacity in their functioning making it difficult to challenge its decisions. This creates what is known as the ‘black box’ problem.[xiii]However, common law legal traditions, where rules develop in a case-by-case manner, allows for nuanced approaches to develop an explainable AI that are in alignment with the demands of transparency and accountability.
As mentioned earlier, larger datasets are required for prediction technologies, which is key determinant in their functioning. A biased dataset results in biased outcomes which is termed as ‘algorithmic bias’.[xiv] Then there is the issue of ‘automation bias’ as well, wherein over-reliance on the AI predicted outcome by judges stifles the organic growth of law. To address these, human review and oversight is necessary. This would ensure that the dynamism necessary for development of law is not hampered. It is, therefore, not pragmatic for algorithmic decision-making to be conclusive but to work under human supervision. The aim should not be to work towards replacing human judgment but at prompting better case triage and action plan for advocates and judges. For instance, recently, the Supreme Court of India launched its AI driven research portal called SUPACE (Supreme Court Portal for Assistance in Court’s Efficiency). The Chief Justice of India while launching the portal emphasised that AI will only function in a way that retain the autonomy and discretion of judges.
Conclusion
Without falling into the trap of techno-solutionism, the need is to capitalise on the growing technologies in the justice administration system with a healthy scepticism which can allow putting in place adequate safeguards. A realistic assessment of the functioning of predictive AIs can be made only after its usage in a particular legal setting. Initialising the process is therefore the key. Early initiation could help develop indigenous paradigms so that there is no need to rely on foreign imports for the same, both in terms of developing technical know-how and principles governing these technologies. This would also provide better contextualisation and develop internal capacity, which would be in aid to the AatmaNirbhar Bharat Mission.
[i](2021) 2 SCC 775.
[ii]Bruno Dondero, Predictive Justice, Professor Bruno Dondero’s Blog(2017), https://brunodondero.com/2017/02/10/la-justice-predictive/.
[iii]Anthony D’Amato, A New (and Better) Interpretation of Holmes’ Prediction Theory of Law, Northwestern University School of Law (2008), https://scholarlycommons.law.northwestern.edu/cgi/viewcontent.cgi?article=1162&context=facultyworkingpapers
[iv] Phoebe C. Ellsworth, Legal Reasoning, University of Michigan Law School (2005), https://repository.law.umich.edu/cgi/viewcontent.cgi?article=1050&context=book_chapters.
[v]Catherne P. Wells, Holmes on Legal Method: The Predictive Theory of Law as an Instance of Scientific Method, Boston College Law School (1994), https://core.ac.uk/download/pdf/80408507.pdf.
[vi]Supra 1.
[vii] Mark K. Osbeck, Lawyer as Soothsayer: Exploring the Important Role of Outcome Prediction in the Practice of Law, University of Michigan Law School (2018), https://repository.law.umich.edu/cgi/viewcontent.cgi?article=3020&context=articles.
[viii]Supra 1.
[ix]Theodore Ruger et al., The Supreme Court Forecasting Project: Legal and Political Science Approaches to Predicting Supreme Court decision making, University of Pennsylvania Carey Law School (2004), https://scholarship.law.upenn.edu/cgi/viewcontent.cgi?article=1671&context=faculty_scholarship.
[x]AI Beats Human Lawyers in CaseCrunch Prediction Showdown + DATA UPDATES, Artificial Lawyer (2017), https://www.artificiallawyer.com/2017/10/28/ai-beats-human-lawyers-in-casecrunch-prediction-showdown/.
[xi] Daniel Fagella, AI in Law and Legal Practice – A Comprehensive View of 35 Current Applications, Emerj (2020), https://emerj.com/ai-sector-overviews/ai-in-law-legal-practice-current-applications/.
[xii] Camilla Ovi et al., The Judge of the Future: Artificial Intelligence and Justice, The European Judicial Training Network (EJTN) (2019), http://www.ejtn.eu/PageFiles/17916/TEAM%20ITALY%20II%20TH%202019%20D.pdf.
[xiii] Ashley Deeks, The Judicial Demand for Explainable Artificial Intelligence, University of Virginia School of Law (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3440723#.
[xiv] Nicol Turner Lee et al., Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms, Brookings (2019), https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and-policies-to-reduce-consumer-harms/.
This article has been authored by Ms. Garima Pal, Research Associate at Centre for Criminology and Victimology, NLU Delhi and Mr. Bhishm Khanna, Associate, Naik Naik & Co. This blog is a part of RSRR’s Excerpts from Experts Blog Series, initiated to bring forth discussion by experts on contemporary legal issues.