Contract AI: How Legal Departments Evaluated, Use Artificial Intelligence Tools

A new report for Seal Software sheds light on how companies’ legal departments are preparing for or using the latest tools associated with artificial intelligence to analyze contracts for hidden opportunities and risks, according to the white paper by Ari Kaplan Advisors.

Seal uses machine learning and other technology to review contracts, extract data and do analytics, the California-based company said in a news release. It all adds up to a “Contract AI” solution.

Most respondents in the study, “Understanding Legal AI from the Inside Out,” said they weren’t using the technology yet, but half of them expected spending on Contract AI to go up this year, the report said.

Law departments tend to seek out this technology and encourage its use, the paper said, adding that Contract AI’s benefits can help other departments that have responsibility for contracts, from drafting to execution.

“The lawyers will care about the discovery aspects, e.g., warranty exposure, regulatory exposure, and GDPR,” one participant said. “Procurement leaders can advise on whether Contract AI is effectively improving their service, e.g., time to value and whether contracts are affecting decision-making. And contract negotiators can inform their drafting processes.”

Some respondents pulled back the curtain on how they’re deploying the technology.

“We use it in the analysis of distribution contracts, which represent about 20% of the contracts we evaluate annually, but our goal is to roll it out worldwide,” said one participant.

Another added: “We use it to help the procurement team extract information in M&A transactions and to support the integration of newly acquired businesses.”

Other applications were listed as:

• Leveraging AI to answer questions from management and other stakeholders.
• Using natural language search to identify provisions in a contract.
• Evaluating third-party supplier documents that are required or prohibited by regulation.

The report also can be used by companies as a benchmark to see how other business have adopted the technology and how they weathered some tough times before enjoying the full benefits of the technology.

“Initially, expectations are great and people are enthusiastic,” one participant said, “but then you soon discover that this new technology will not solve all of your problems because you need human power to support it, which results in a second level of disappointment. Don’t stop after the second disappointment phase because it is a worthwhile investment.”

Another respondent said training is key.

“The accuracy after training was significantly better than our proof of concept period so avoid judging the system too early in the process because you might underestimate its value,” the respondent said. “In general, the pitfalls of using the AI all come down to the data quality and extent of your training.”

The paper, which can be downloaded here, has other practical advice on how to vet a new system and encourage your staff to adopt it.

“Perform a side-by-side comparison of humans using it and humans who are not so that you can properly measure against your KPIs. If you
do not do this comparison, you will overestimate the accuracy of your humans and underestimate the accuracy
of the technology. This is particularly relevant in the legal field where attorneys think they are perfect and never make mistakes.”

Another respondent said “the most common misimpression is that it will work out of the box, but you need to train your team and the tool to make your use case truly viable.”

The process to secure approval for a pilot project took up to three months, the report said.