- Jan 6, 2017
- 835
Tech beats people in study by AI platform and law experts.
A new study conducted by legal AI platform LawGeex in conjunction with law experts from prestigious universities across the USA threw experienced lawyers against a specialist type of AI trained to evaluate legal contracts, and humanity lost.
The landmark study saw 20 experienced US-trained lawyers pitted against the LawGeex Artificial Intelligence algorithm. The results seem to clearly show that for the first time, AI is comprehensively more accurate at spotting risks, at least where Non-Disclosure-Agreements (NDAs) are involved.
While perhaps few people would be surprised at the results; that Artificial Intelligence works faster than lawyers on some specific legal tasks, the results at first glance appear very close.
“What do you mean your new lawyer only charged you for 30 seconds of their time?”
And the winner is…
The AI finished the challenge with an average 94 percent accuracy, while the panel of expert and experienced attorneys had an average accuracy rate of 85 percent.
The human lawyers, under scientific testing conditions, were given four hours to review five non-disclosure agreements (NDAs) and identify as best they could, 30 legal issues including arbitration, confidentiality of relationship, and indemnification. The lawyers were scored by how accurately they identified each issue.
The lawyers took 92 minutes on average to review and complete the task. The AI however took a mere 26 seconds on average.
Crucially, it’s that distinction in time that is groundbreaking. Not only was the AI more accurate than its human opponents, it was 212 times faster.
Every man and every day
According to the authors of the study, the task was very similar to what many lawyers do every day.
The results, if they successfully pass the peer-review process, could finally put an end to the idea prevalent among most legal professionals and the public in general that machines are unable to match or surpass human skill and experience in fundamental routine legal work.
The LawGeex AI was trained using tens of thousands of NDAs, utilizing custom-built machine learning and deep learning, which allowed it to distinguish between different legal concepts.
Why is this test different from all the others?
The LawGeex AI challenge differs in a number of important respects than any previous such study. For a start, the study made sure to ensure significant expertise of the lawyers in the exact area of law it was assessing.
The study also ensured that all participants carried out precisely the same task. Dr. Roland Vogl, Lecturer in Law at Stanford Law School, and consultant on this project, said: “The Robo-lawyer really has two faces. One is mechanizing and automating legal services – commoditized legal work and document checking. Legal prediction is the other aspect. This test falls in line with the first type of AI automation.”
One of the key outcomes of the study is that AI won’t be taking over from human lawyers anytime soon… I guess you can’t have everything.
A new study conducted by legal AI platform LawGeex in conjunction with law experts from prestigious universities across the USA threw experienced lawyers against a specialist type of AI trained to evaluate legal contracts, and humanity lost.
The landmark study saw 20 experienced US-trained lawyers pitted against the LawGeex Artificial Intelligence algorithm. The results seem to clearly show that for the first time, AI is comprehensively more accurate at spotting risks, at least where Non-Disclosure-Agreements (NDAs) are involved.
While perhaps few people would be surprised at the results; that Artificial Intelligence works faster than lawyers on some specific legal tasks, the results at first glance appear very close.
“What do you mean your new lawyer only charged you for 30 seconds of their time?”
And the winner is…
The AI finished the challenge with an average 94 percent accuracy, while the panel of expert and experienced attorneys had an average accuracy rate of 85 percent.
The human lawyers, under scientific testing conditions, were given four hours to review five non-disclosure agreements (NDAs) and identify as best they could, 30 legal issues including arbitration, confidentiality of relationship, and indemnification. The lawyers were scored by how accurately they identified each issue.
The lawyers took 92 minutes on average to review and complete the task. The AI however took a mere 26 seconds on average.
Crucially, it’s that distinction in time that is groundbreaking. Not only was the AI more accurate than its human opponents, it was 212 times faster.
Every man and every day
According to the authors of the study, the task was very similar to what many lawyers do every day.
The results, if they successfully pass the peer-review process, could finally put an end to the idea prevalent among most legal professionals and the public in general that machines are unable to match or surpass human skill and experience in fundamental routine legal work.
The LawGeex AI was trained using tens of thousands of NDAs, utilizing custom-built machine learning and deep learning, which allowed it to distinguish between different legal concepts.
Why is this test different from all the others?
The LawGeex AI challenge differs in a number of important respects than any previous such study. For a start, the study made sure to ensure significant expertise of the lawyers in the exact area of law it was assessing.
The study also ensured that all participants carried out precisely the same task. Dr. Roland Vogl, Lecturer in Law at Stanford Law School, and consultant on this project, said: “The Robo-lawyer really has two faces. One is mechanizing and automating legal services – commoditized legal work and document checking. Legal prediction is the other aspect. This test falls in line with the first type of AI automation.”
One of the key outcomes of the study is that AI won’t be taking over from human lawyers anytime soon… I guess you can’t have everything.
