By DebbieH 07 Nov 2017 7 min read

100 lawyers lose to a robot in legal competition

CaseCrunch Alpha, an artificially intelligent system, went up against more than 100 commercial lawyers from London to predict the outcome of mis-sold PPI cases. These were real cases and both the lawyers and the AI program were asked to predict whether the complaints were upheld or rejected by the Financial Ombudsman Service. Over 750 predictions were made and the robot outperformed the lawyers by a considerable margin. The lawyers had an accuracy of 62.3% and CaseCruncher scored 86.6%.

CaseCrunch say that their AI performed so much better than the lawyers because it ‘had a better grasp of the important of non-legal factors than lawyers’. A research paper has been promised which will take a deeper look at their findings. They were careful to add that the question must be ‘defined precisely’ for the AI to reach such impressive levels of accuracy.

This is the project of Cambridge graduates Joseph Maruscak, Rebecca Agliolo, and Ludwig Bull. Although none of them have a background in computer science and all read law at university, Scientific Director Ludwig Bull taught himself about AI. He took remote courses offered by Stanford University and the students began their project by developing a chatbot capable of answering legal questions. This has been transformed into the more sophisticated CaseCruncher Alpha used in the competition vs. the human lawyers.

Despite this AI still being in its early stages, questions have arisen as to whether this will be the sort of technology that helps or potentially replaces junior lawyers. The BBC reported the opinion of Ian Dodd who sums up one view succinctly; “The knowledge jobs will go, the wisdom jobs will stay”.

This comes at a time when there has been much discussion about the potential role AI could play in the future of the legal industry. The Law Society published an economic forecast last week which suggests that Artificial Intelligence and the automation of tasks could ‘transform the legal sector’. They predict that 20% of legal jobs could be lost by 2038 due to the progress of AI. This is expected to be largely offset by a rising demand for legal services.

Deloitte published a report at the beginning of last year which found that over 100,000 of the roles in the legal sector could be automated in the long term. However, as the report explains, despite the loss of lower-skilled jobs, highly-skilled roles will have to be created to manage and develop this new technology.

This highlights the fact that the roles that are most likely to be affected by AI are administrative roles. Some of the tasks usually carried out by junior associates, such as research tasks, are already being automated. The world’s first autonomous robot lawyer, DoNotPay, has helped win hundreds of thousands of cases against parking tickets. It seems that AI has arrived in the legal sector and has the potential to streamline the industry.

There are several companies that have appeared recently who are developing AI to help lawyers become more efficient. Here are a few to watch:

Ross Intelligence: This company from California have developed an artificially intelligent system that learns as it is used.

LawGeex: This AI can review contracts and highlights issues based on your business’s legal policies.

LegalSifter: Provides information on Terms and Conditions or NDAs to highlight parts that need adjusting.

TrademarkVision: Uses machine learning and image recognition to search for similar trademark images already in use.

Counselytics: This firm from New York City uses AI to analyse large amounts of legal data to provide valuable insights.


Stay up to date with the latest legal news with the Simply Law Jobs blog.