FRONTEO Inc. (Headquarters: Minato-ku, Tokyo, President: Masahiro Morimoto, hereinafter FRONTEO) will implement it on the AI ​​review tool "KIBIT Automator" equipped with the AI ​​engine "KIBIT" developed in-house. , Has succeeded in developing a new index useful for quantitative evaluation of the reviewer's work quality.This index automatically measures business quality numerically using AI, and enables effective management and measures to improve business quality.

　The amount of data managed by companies is steadily increasing year by year, and in digital forensics (information security and analysis surveys targeting information recorded on digital devices), several TB per evidence holder (Castdian) You have to collect data that can range up to, and find evidence-related documents in a vast amount of documents within a limited period of time.Time, cost, and survey quality are major challenges for lawyers involved in the survey.Among them, document review, which is the process of discovering related information, is said to account for 1% of the time and cost, and the use of AI is indispensable. KIBIT Automator is already in operation in the United States and Japan in the legal tech field, and the high-performance AI engine KIBIT, which has learned the judgment of lawyers, classifies documents related to evidence and documents not related to evidence. It significantly reduces the amount of documents that need to be reviewed by, and saves time and money.

　FRONTEO has focused on improving the quality of reviews by humans, and has worked on the development of evaluation indicators, recognizing that it is important to manage the work quality of reviewers appropriately numerically.This evaluation has traditionally been done using the percentage of the number of documents in charge called the Overturn Rate, which was found to be incorrect in the document classification by re-review.However, in this method, for example, in an environment where conditions other than the number of documents in charge (document type, difficulty level, rate of including documents related to evidence) are the same, 20 out of 5 documents make mistakes. If there is a mistake in 200 of the 50 documents, the latter should be judged to have higher reliability and more mistakes, but in both cases it is evaluated as the same number of 25%, etc. There was a problem.Therefore, FRONTEO statistically treats the measurable numerical value group in the document review operation and corrects it to the ratio of the number of missed documents to the number of documents that have undergone quality control of document review called Quality Control (hereinafter referred to as QC). By doing so, we solved the above problems and created a new evaluation index.By automatically calculating this evaluation index with our proprietary AI, it has become possible to more accurately evaluate the review quality of reviewers in document reviews.

　In the above example, in the new method, the latter is three times more problematic than the former.However, if the cause of the mistake of 3 documents is due to the large number of QC documents, it will be automatically corrected to a value smaller than 50 times.As a result, candidates for important reviewers in managing and improving the quality of document reviews, such as accurate (low Overturn Rate) QC reviewers who perform QC and attention reviewers who have problems with accuracy (high Overturn Rate), are selected. , You can select more correctly.

　Figure XNUMX shows a comparison of the Overturn Rate actually calculated by the conventional method and the new method using FRONTEO test data.It has become clear that the new method can analyze and automatically select QC reviewers and reviewers to be watched more appropriately than the conventional method.In this way, by using more statistically valid numerical indicators, AI can analytically support the proper selection and evaluation of reviewers, further promoting the management of reviewers and the improvement of review quality. increase.

Figure 2.Comparison of Overturn Rate calculated by the conventional method and the new method (using FRONTEO test data).The boxed part of the graph is an example where the reviewer who seems to be selected as a quality control (QC) person with high quality and the reviewer with low quality (many mistakes) who is a candidate for attention changes between the two methods. Is shown.

　As a pioneer of digital forensics and discovery (discovery procedures in the US civil litigation system) in Japan, FRONTEO will continue to strive to develop and improve solutions that help streamline fraud investigations and litigation support.

■ About KIBIT Automator URL: https://legal.fronteo.com/products/kibit-automator/
"KIBIT Automator" improves the efficiency of document review work in electronic discovery (e-discovery), reduces the burden on workers, and reduces costs, among other discoveries required in the trial procedure of US civil proceedings. An AI tool developed for the purpose, released in March 2019.Applying the research method used in discovery, AI is used to examine and analyze large-capacity e-mails and electronic files that are evidence materials.In recent years, it has been utilized as one of the important processes of digital forensic surveys in the surveys of third-party committees in Japan, and it is expected to respond to the short-term information disclosure required by companies.

■ About KIBIT　URL: https://www.fronteo.com/products/kibit/
"KIBIT" is an artificial intelligence that analyzes text without relying on keywords, using a unique machine learning algorithm that reproduces the "tacit knowledge" possessed by specialists and business experts.Highly accurate analysis in a short time is possible with a small amount of teacher data.

■ About FRONTEO　URL:https://www.fronteo.com/
FRONTEO uses the in-house developed AI engines "KIBIT", "Concept Encoder (trademark: conceptencoder, reading: concept encoder)", and "Looca Cross", which are specialized in natural language processing. It is a data analysis company that supports the business of companies by extracting meaningful and important information from a huge amount of text data. Since its establishment in August 2003, it has been expanding globally to Japan, the United States, South Korea, and Taiwan, focusing on legal tech businesses such as "e-discovery (electronic discovery)" and "digital forensic investigation" that support corporate international litigation. Has been deployed.Based on the AI ​​technology cultivated in this business, we will expand the business field to the life science field, business intelligence field, and economic security from 8, and by using AI to "turn text data into knowledge", We contribute to solving various corporate issues such as drug discovery support, dementia diagnosis support, financial, personnel, and sales support. Listed on TSE Mothers (currently TSE Growth) on June 2014, 2007. Obtained a manufacturing and sales license for first-class medical care in January 6 (permit number: 26B2021X1), and notified the managed medical device sales business in September of the same year (notification number: 13 Minato Misei Equipment No. 1).The capital is 10350 thousand yen (as of March 9, 3).

* FRONTEO, KIBIT, conceptencoder, and Looca Cross are registered trademarks of FRONTEO in Japan.