📄 Please take our evaluation survey to help us improve the site!
highlighted
to the right, which point to the interconnectedness of many use cases.The legal tech department of a German law firm with over 350 employees is exploring the usage of AI-based solutions to support the daily legal work of the lawyers. The department consists of 6 employees and is responsible for the digital transformation and legal tech strategy.
The most important problem to solve is how to schematically process large collections of documents. Therefore, extraction of relevant information from documents is the main use case the department is using AI-based solutions for. Other related use cases are the classification of documents and being able to ask the AI questions about the contents of a document.
The law firm has been using tools based on machine learning for multiple years and is currently looking at solutions like Legisway Analyzer (https://www.wolterskluwer.com/en-gb/solutions/legisway/legisway-analyzer) and Harvey (https://www.harvey.ai/). The output is perceived to be really good but still requires manual checking to make sure the information is not hallucinated.
Production - users are using the tool in their daily workflow
Two interviews conducted in May 2024, led by sebis researchers, one interviewee is a lawyer and the head of the legal tech department at the company, the other is an associate of the same law firm and part of the legal tech team.
The solution is targeted at small- to medium-sized law firms.
Mioto Labs (https://mioto.app/) offers custom-tailored solutions for customers in the legal domain. Typical use cases include but are not limited to contract generation based on pre-defined templates, document review and analysis, and information extraction.
The solution currently consists of a workflow builder for process automation. Commercial LLMs like ChatGPT through an Azure instance are used to assist in specific steps of the workflow, e.g., to extract information from documents. Depending on the needs of the customers, the team decided where and when in the workflow chain it makes sense to utilize LLMs to achieve a specific subtask.
Interview conducted in May 2024, led by sebis researchers. The interviewee is a co-founder and the CTO of the company.
The corporate Merger & Acquistions department of a law firm totaling over 100 employees and specializing on German law is testing the usage of AI and LLMs to speed up their work.
The employees use an LLM-based tool in hopes of improving their efficiency to draft contracts. Normally, new contracts are constructed by reusing and adapting clauses from past contracts. The tool helps automate this task by importing clauses from other documents and adapting them to the new context. The tool is used for assistance as it still requires a good amount of manual effort. One reason for incorporating AI technology into their weekly routine is the perception that customers now expect it in this day and age.
A few selected lawyers of the department are testing Henchman (https://henchman.io/), a multi-LLM contract drafting and negotiating AI solution, to assess a potential department-wide adoption of the tool. One of the employed LLMs is ChatGPT running on an Azure instance so that the data is not used to train the model and remains within the EU. Users can interact with the system through an Add-In in Word or Outlook.
Research - different users are experimenting with the usage of LLMs
Two interviews conducted in April and May 2024, led by sebis researchers. One interviewee is a lawyer working in corporate Mergers and Acquisitions at a German law firm and the other is a legal tech consultant.
A large German law firm is developing their own solution to assist in their legal work but the solution is also provided to interested legal professionals who need to generate documents in their daily work, e.g., contracts.
The main use case aims at generating legal documents based on templates and data inputs but the solution is also able to assist in other legal tasks surrounding document management. The company is currently evaluating the usage of AI in various aspects of their work to derive new use cases that can benefit from its usage.
The company uses AI pipelines (i.e., Retrieval Augmented Generation) that are independent of a specific LLM and always aim to use the best LLM depending on use case. Currently, they are working with GPT-4 deployed on their own Azure instance but are aiming at also including Claude Opus. The company took all their documents, anonymized them, and embedded them in a vector database to allow for a semantic search. From this existing knowledge base new documents can be generated through pre-defined prompts. The functionality is accessible through a Word Add-In that is connected to the pipeline built on the deepset cloud. Subtasks of the pipeline also employ other NLP methods such as Named Entity Recognition (NER).
Research - the solution is under active development and different users are evaluating its usage
Interview conducted in April 2024, led by sebis researchers. The interviewee is the CEO of a spin-off company.
A large law firm with over 300 lawyers is evaluating the use of legal AI at their firm. The target audience of the tools are legal professionals.
The firm is interested in understanding the potential benefits of using AI in their practice, such as automating repetitive tasks, improving decision-making speed, and increasing overall efficiency. Therefore, the firm called into action an AI initiative consisting of multiple lawyers to explore and evaluate the use of legal AI.
The use cases under investigation include:
For Question Answering, they are building a curated knowledge base containing all admissible company data that can then be queried through the chat interface. The semantic search also provides references to the sourced documents.
The law firm is evaluating the use of different LLMs including proprietary models like ChatGPT (OpenAI) and Luminous (Aleph Alpha), as well as open-source models like Llama. For their daily work, all employees have access to ChatGPT deployed on their own Azure instance to ensure data is not used for training. To manage and reduce the costs associated with LLM usage, token length is limited. Some users are also testing Copilot for their daily work needs.
Production - different users are utilizing AI technology regularly in their workflows
Interview conducted in March 2024, led by sebis researchers. The two interviewees are both lawyers at the law firm.
The solution is targeted at law firms and legal professionals.
The main focus is to provide transparency regarding legal case data, i.e., every legal case is self-contained and builds the foundation that the software solution operates upon. Users can query the documents for information and ask about information related to each case.
The company provides an AI-based case analysis tool called Herlock.ai (https://www.herlock.ai/) that uses natural language processing (NLP) and vector embeddings to analyze legal case data. The solution is provided as SaaS deployed on a private cloud running on GCP or AWS. One of the used language models is ChatGPT but the company is looking at alternatives that can be deployed on-premise for customers that have this requirement. The solution organizes relevant information around individual legal cases and provides a visual timeline of chronologically sorted events next to allowing to search and compare documents and ask questions about them.
Interview conducted in April 2024, led by sebis researchers. The interviewee is the co-founder and CTO of the company.
The employees of a large German accounting firm with over 15.000 employees are exploring the usage of LLM-based tools to improve their efficiency.
The employees use an LLM-based tool for document analysis, contract drafting, negotation assistance, and legal research. The most important use case for the users is document summarization, e.g., of contracts.
The employees are using Leah offered by ContractPodAI (https://contractpodai.com/leah/). ContractPodAI provides a contract lifecycle management suite and Leah is a standalone SaaS solution that enables the usage of generative AI without having to implement the full CLM solution. Leah provides various functionality out-of-the-box and utilizes multiple LLMs under the hood to generate its responses but it can also be fine-tuned to the needs of the customers.
Production - users are using the tool in their daily workflow
Interview conducted in May 2024, led by sebis researchers. The interviewee is the head of legal tech at the company.
The High Court of the Canton of Aargau in Switzerland with overall roughly 470 employees of which 124 are using an anonymization solution.
The court wanted to publish court decisions but needed to anonymize sensitive data before being able to do so. Therefore, they chose Balo.ai (https://www.balo.ai) which allows for the anonymization of text documents while preserving the readability of the document and keeping all data on-premise. Through the usage of the anonymization solution, the court was able to publish a lot more decisions than before (1673 in one year as opposed to roughly 100 in the past).
Balo.ai works with an on-premise language model so that no data ever leaves the environment the tool is used in, which is particularly beneficial from a data protection perspective as the target users are mostly public institutions in the legal sector who operate under strict data protection regulations. The model is specifically trained for anonymization tasks and doesn't require a lot of computing resources, which makes it possible to host it on machines with limited resources. Additionally, there is a MS Word Add-In that makes anonymization suggestions that the user needs to manually accept and a separate tool for PDFs with the same functionality.
Production - the courts are using the solution for anonymization
Two interviews conducted in April and May 2024, led by sebis researchers, one with a co-founder of Balo.ai and one with a test manager working for the courts of the Canton of Aargau.
The solution is targeted at public institutions like the government, courts, and other regulatory offices. The company is currently in contact with the justice ministry of Bavaria to explore how the solution can provide value to them.
The company Elephant Labs (https://www.elephantlabs.ai/) was founded in December 2022 in an attempt to develop a solution that increases the efficiency of lawyers through the use of computer science. For example, court verdicts need to be published to the public but the regulations as to how and when are not clear. In order to publish such data, however, sensitive information needs to be masked first and this is what VerdictLLM was initially developed for.
The tool VerdictLLM utilizes the power of open-source LLMs and statistical methods to anonymize sensitive data. Additionally, the company uses embeddings to classify which information needs to be anonymized. The LLM is fine-tuned through synthetic data specifically optimized for applicability in the legal context.
Interview conducted in April 2024, led by sebis researchers. The interviewee is the founder of the company.
A German law firm with roughly 180 employees has established a small legal tech lab and is exploring the usage of AI-based solutions to support their daily legal work. The department consists of 6 employees and is responsible for the digital transformation and legal tech strategy.
The employees are trying to improve their knowledge management. The main goal is to make all relevant documents and data accessible through semantic search so that lawyers can easily find what they are looking for and then query these documents and receive appropriate responses from the AI. One example is to upload a document and ask for a summary of the decision. If the answer is not specific enough, the lawyer will ask a follow-up question to clear up any uncertainties. The users perceived big improvements in their efficiency as the solution is able to assist them in finding relevant information quicker than when looking for it themselves, especially considering that some cases consist of many documents which in turn consist of many pages. As LLMs may hallucinate responses, all users are urged to double check the generated responses.
The law firm is currently integrating Prime Legal AI (https://primelegal.de/en/) into their workflow. Depending on the use case, the AI solution utilizes multiple already existing LLMs (e.g., GPT-4, GPT-4-Turbo, Claude-3-Opus) under the hood to provide its various services. The users can select from a library of pre-defined prompts (or write their own) and the data they want them used upon. The anonymization functionality works on-premise whereas the rest of the functionality is available through the cloud. The service is accessible through a web application but also through an API. An entity-relationship model was used to train knowledge graphs which are the foundation for Prime Legal AI.
Production - users are using the tool in their daily workflow
Two interviews conducted in May 2024, led by sebis researchers, one interviewee is the CEO of Prime Legal AI and the other is a lawyer specializing in corporate law and responsible for legal tech at their law firm.
The patent department of a global technology and engineering company, which includes 26 direct employees and around 200 in total, is testing the usage of AI and LLMs to speed up their work.
The employees use the LLM for various tasks surrounding text generation and analysis. Since September 2023, they are testing what is possible concerning patents including patent drafting, mapping of patents, and patent analysis, i.e., retrieving key aspects out of the documents. The results are perceived to be quite promising but one still needs to invest some time refining them as they are not good enough to just be copy pasted and used as is.
The initial expectation was that you can just give the LLM a job and wait for it to come back with the results. However, one key learning is that you need to teach the model what you want, i.e., give it a persona and break down the instructions into very small parts.
Further use cases revolve around assistance in their daily jobs such as drafting emails or parts of reports and translations.
In the future, they would like to expand their usage of LLMs by leveraging the voice input capabilities. Ideally, the LLM will act as a personal assistant helping with their daily tasks, e.g., answering questions, providing explanations, finding and opening files, etc.
The employees have access to ChatGPT 3.5 with 128k tokens through the company's Azure instance which provides a certain level of security and confidentiality. The department is considering the costs and benefits of prompt optimization or the potential fine-tuning or even training of own models.
Research - different users are experimenting with the usage of LLMs
Interview conducted in June 2024, led by sebis researchers. The interviewee is a lawyer and head of the patent department at the abovementioned company.
The solution is targeted at law firms and notaries working with land registration data.
The company fastlaw (https://fastlaw.online/) was founded as a spin-off from a law firm that has been working completely digitally for the past ten years alreay. Their AI solution is designed to read land register data and extract relevant information.
The company provides a CRM solution for law firms and an AI-based tool called GREGOR available as a SaaS solution that is able to read land register data through OCR and then extract relevant information which in turn can be exported again in a structured format. To achieve this, the company together with an undisclosed partner trained their own language model.
Interview conducted in April 2024, led by sebis researchers. The interviewee is the head of legal engineering and business development of the company.
The solution is targeted at law firms and legal professionals.
Libra (https://libratech.ai/) wants to become an AI-powered law firm assistance provider to increase the efficiency of lawyers. For example, the tool is able to classify and extract information from documents and generate new documents like contracts.
The company is currently building modules in an AI pipeline to automate different tasks in a lawyer's workflow. They started out with using commercial APIs like ChatGPT but then proceeded to fine-tune their own models that will be used exclusively in the future. The solution is under active development and the company is building their own evaluation data sets while simultaneously receiving feedback from customers to evaluate their application.
Interview conducted in April 2024, led by sebis researchers. The interviewee is a co-founder and the CTO of the company.
The Mergers and Acquistions department of a large international law firm with roughly 70 lawyers in Germany (900 globally) is exploring the usage of AI in their daily work. The law firm has a global technology committee consisting of multiple employees including the CIO that evaluates new technologies and makes executive decisions about adopting them into the firm.
The employees are using AI-based tools for document analysis, information extraction, and translation. They believe AI tools will eventually replace a lot of the work that is nowadays being done by junior associates.
The department is using Kira (https://kirasystems.com/) for document analysis and information extraction from legal documents like contracts. The firm has access to its own instance running the software to ensure the confidentiality of the data. For translations the lawyers use DeepL (https://www.deepl.com/), an AI-based translation tool specifically trained for translation tasks. Furthermore, the firm also tested ChatGPT but deemed it not good enough to be used as-is in their daily professional work and is now evaluating Harvey (https://www.harvey.ai/) for document analysis.
Production - AI-based tools are used regularly in their legal work
Interview conducted in February 2024, led by sebis researchers. The interviewee is a lawyer working in Mergers and Acquistions at a large law firm.
A large publishing group with its own legal department consisting of 6 lawyers is evaluating the usage of AI tools for their daily legal work.
The employees have access to a company-wide chatbot that was developed in-house and that they can use for their daily work. The use cases include:
The company chatbot is based on models from OpenAI but does not have access to the internet or company data directly. However, users can upload documents through the chatbot interface to use them in ensuing prompts. The company is also testing Semantha (https://www.semantha.de/) for document analysis and semantic search.
The legal professionals do not have a predefined set of evaluation metrics but come together regularly to discuss the usage of AI and other relevant topics revolving around AI. The general sentiment is that there is a lot of potential in AI solutions but we are still just at the beginning and the tools currently do not work satisfactorily enough yet.
Production - employees use company-wide chatbot for their daily work
Interview conducted in March 2024, led by Prof. Dr. Florian Matthes and Nektarios Machner. The interviewee is working at the firm's legal department as a lawyer with a focus on IT.
A large law firm with over 100 lawyers and accountants organized in teams of 10-12 employees distributed over 3 locations is in the process of evaluating legal AI usage at their firm. The target audience of the tools are the lawyers. As of the date of the interview, the law firm has not yet adopted any company-wide legal AI solution but is testing various tools and models in a decentralized way to determine in which cases an adoption is feasible. Some lawyers already use personal ChatGPT accounts to assist them in their daily tasks.
The employees are in the phase of testing and figuring out what problems related to the legal domain can feasibly be solved with the usage of LLM-based tools. Some lawyers already use personal ChatGPT accounts as a personal assistant to assist them in their daily tasks which are not restricted to the legal domain such as creating and summarizing texts (e.g., emails) or helping with translations, etc. Regarding the legal use cases, due to the decentralized nature of the law firm the respective tasks of each department are diverse. The use cases under investigation include but are not limited to:
The law firm is looking at various tools on the market, especially ones also being used by other German law firms, to get an overview of the current tool landscape. The solutions under investigation mainly operate with OpenAI and LLaMa models and are either deployed in the cloud or on-premise. In general, the law firm would prefer an on-premise solution to maintain control over the service. The language requirements are German and English. There is no intention of building their own tools but rather wait for the establishment of market standards that can then be adopted into their own IT landscape.
Since the rise of LLMs there have been multiple individual uncoordinated tests of LLM usage on a personal level but in the past few months an effort has been made to coordinate the testing and conduct it in a more organized way. For this purpose, the various departments of the law firm individually test tools regarding their feasibility for their respective use cases. The employees do not use any specific evaluation metrics but come together and define success criteria and test scenarios for each task and then later on get together in working groups to discuss the findings.
No privacy-sensitive data is used for testing to avoid any privacy-related issues.
Research - different users are experimenting with the usage of AI
Interview conducted in February 2024, led by sebis researchers. The interviewees are working at the law firm in the position of lawyer and Digital Transformation Manager respectively.
The solution is targeted at law firms and legal professionals.
The motivation to focus on the legal sector stems from the fact that legal professionals primarily work with text which is predestined to being used with large language models. The offered AI suite aims to assist with multiple legal use cases including case management, document analysis, and document generation based on custom templates specific to the customer.
Hyde (https://www.hyde.to/) is an AI suite based on language models. The company transitioned from building one large language model into building multiple smaller models specific for individual use cases such as semantic search to query documents for information. The solution is offered as SaaS in the cloud or on-premise depending on the customer's need.
Interview conducted in May 2024, led by sebis researchers. The interviewee is a late co-founder and the CPO at the company.
The solution is targeted at law firms and legal professionals.
The company focuses on data collection and data quality to enable digitalization. Clients of law firms can use the solution to provide relevant data for their case in a structured form by following the steps in the tool. Lawyers are then able to review the data digitally and ask questions about the document to the AI assistant.
Justin Legal (https://justin-legal.com/) provides their service through a SaaS web application. The underlying LLM is ChatGPT 4 deployed on an Azure instance which is used to provide features like text summarization and question & answering. The data can be used to generate new documents, export in various formats, or directly passed via API to standard software solutions.
Interview conducted in May 2024, led by sebis researchers. The interviewee is the CEO at the company.
The solution is targeted at professionals from the legal and financial sector.
The company Neur.on AI (https://neur-on.ai/) specializes in translations and translation management.
The company currently provides multiple separate solutions like Lex Machina and Corrext that they eventually want to integrate into one overarching software solution. The solution is offered as a collaborate SaaS solution accessible through a web application. The system preprocesses data with the help of multiple LLMs, e.g., to categorize what kind of law documents pertain to, and then the system can provide machine translations but also humans can collaborately check the results and work on creating the translations through a web interface manually. The models were trained specifically for legal and financial terminology.
Interview conducted in May 2024, led by sebis researchers. The interviewee is the COO at the company.
The legal tech department of a German law firm is developing and using an AI-based solution for their daily legal work. The department is responsible for legal operations and tries to model legal processes as a series of steps that can be automated with software solutions.
The motivation for the solution was an ever increasing influx of legal correspondence that took too much time sorting through and properly linking it to the respective cases. Using an AI-based solution to go over all incoming correspondence automatically improves their efficiency since documents are classified and information like deadlines is extracted automatically.
The law firm uses Salesforce as CRM where they integrated their AI solution through multiple complex workflows. The solution Blaubach.ai (https://www.blaubach.ai/) was specifically developed for internal use first but is now also offered to other interested parties and can be adapted to their specific needs. The general workflow is that incoming data is passed through the AI solution that automatically detects the docket number and extracts relevant information from the document assigning it to the correct case. The lawyers annotated the training data themselves and handed the labels off to the data scientists in the team to adapt the AI.
Production - users are using the tool in their daily workflow
Two interviews conducted in April 2024, led by sebis researchers, one interviewee is a lawyer and the head of the legal tech department at the company, the other is a data scientist.
A team of several in-house legal experts from a multi-national, German-based technical corporation specializing in healthcare solutions and services. 300 users are currently testing the usage of MS Copilot.
The employees use LLM-based tools for various tasks in their daily work life. For one as a personal assistant for various tasks including quick general questions, translations, summaries of text, and searching for specific files on the system. Another task can be summed up under the umbrella term of content generation, e.g. for email drafts, marketing texts, etc.
The rise in popularity of ChatGPT led legal experts at the company to evaluate if they could also harness the power of LLMs to assist in the daily, routine work of legal contract generation and analysis, which typically requires a great deal of manual time and human effort. The usage of LLMs can help highlight important parts of contracts and give guidance on where a human expert should have a closer look. The contracts are based primarily on German law. Other legal tasks are also being considered.
Another emerging use case is the usage of LLMs to assist in software development wherefore it's currently being tested regarding legal regulations.
Finally, the organization is thinking about deploying chatbots to assist in the communication with customers which could result in a reduction of costs.
As a technical-oriented organization, employees already had existing access to the Microsoft Office products, which facilitated for the convenient experimentation with MS Copilot and ChatGPT on the Microsoft Azure Cloud. This cloud solution was chosen due to its ease of use, as well as the perceived privacy of using ChatGPT in the cloud, with no need for further training or fine-tuning on proprietary data. Prompt engineering is employed to optimize results.
Software engineers are also evaluating open-source models for generating code and documenting it automatically.
The organization started using the ChatGPT playground in summer 2022. Before that, interested parties experimented with their personal ChatGPT accounts and non-sensitive data, subject to guidelines provided by the organization. The employees do not employ any prompt engineering guides on department level or use any evaluation metrics to determine the effectiveness of the model. The experts manually test the models for their various tasks to determine if their usage for the respective use cases makes sense. One finding is that the more the respective field is legally regulated, the less sense it makes to employ AI, e.g. German lease contracts are too strictly regulated so that LLMs struggle with producing accurate and correct results.
Documents are anonymized manually before uploading them to the cloud and making them accessible to the model. Usage of the public ChatGPT interface is only allowed for non-privacy sensitive matters. For everything else the ChatGPT playground needs to be used as Microsoft claims to not use that data for training purposes.
Research - different users are experimenting with the usage of LLMs
Interview conducted in January 2024, led by Prof. Florian Matthes and Nektarios Machner. The interviewee is a lawyer and responsible for the IT department at the abovementioned company, where he also looks into what is going on in the IT world regarding useful tools.
The solution is targeted at legal service providers.
913.ai (https://www.913.ai/) wants to provide an automation infrastructure that allows users (e.g., legal engineers) to connect various already existing tools in a law firm with each other. Use cases include but are not limited to document classification and information extraction. The solution aims to represent and assist in typical workflows in the daily legal work of lawyers.
913.ai uses open-source LLMs but also fine-tunes their own models for specific tasks like text extraction or classification. They offer a pipeline builder that allows users to select and link various modules together to depict their own unique scenarios as they need them. They are currently adding more modules and improving already existing ones to make them more adaptable by the users.
Interview conducted in April 2024, led by sebis researchers. The interviewee is a lawyer, co-founder and the CEO of the company.
A German law firm with roughly 120 lawyers is exploring the use of AI in their daily legal work. The firm put together an AI Task Force consisting of multiple lawyers from the firm to evaluate AI-based tools regarding their applicability in their daily workflow.
The main use cases are document analysis and document generation, e.g., for documents submitted by a party in a legal proceeding or to create pitch decks. Further use cases include translation and general research. To assist with cases concerning intellectual property, an AI-based tool is used to do similarity analysis, e.g., comparing two logos or design.
Overview of use cases:
The law firm is currently evaluating ChatGPT to find out what for and how it can be used in their legal work. Additionally, Microsoft's Copilot is being explored as a personal assistant, e.g., to search for specific files on the file system. DeepL Pro (https://www.deepl.com/), an AI-based translation tool specifically trained for translation tasks, is used for translations.
No privacy-sensitive data is used for testing to avoid any privacy-related issues.
Production - lawyers use AI-based tools for their daily legal work
Interview conducted in February 2024, led by sebis researchers. The interviewees are working as lawyers at the law firm.
About Us | Contact Us | © sebis 2024 | sebis Homepage | Imprint