With the advent of ChatGPT, AI has come to the attention of not only researchers and developers, but also the general public. AI has been developed and put to practical use, and companies and researchers around the world are still working hard to develop AI for all kinds of purposes.
Is AI, including ChatGPT, a "dream tool" beyond human knowledge? We will explain through familiar examples and AI features.
FRONTEO Corporation
Director/CTO
Doctor of Science
Hiroyoshi Toyoshiba
D. in Mathematics. Engaged in research on gene expression data analysis, target discovery, biomarker discovery, etc. at the National Institute of Environmental Health Sciences (NIEHS) and Takeda Pharmaceutical Company, etc. He researched and developed FRONTEO's AI algorithms.
AI stands for Artificial Intelligence. Although "AI" is no longer a common noun, there is still actually no clear unified definition of the term. In a wide range of fields, it may be defined as something that perfectly mimics human intelligence, intelligence, and emotion. Generally speaking, however, AI is more often a system that is programmed to analyze data and learn to perform specific tasks that have been performed by humans in the past*. For example, there are a variety of AIs around us. ChatGPT, which enables written communication, self-driving cars, voice assistants (smart speakers), and facial recognition systems are also AI-based services.
* AI that perfectly mimics human intelligence in the former category is sometimes classified as "strong AI," while AI that performs specific tasks in the latter category is sometimes classified as "weak AI. All AIs currently in practical use are weak AIs.
For example, ChatGPT is a very useful AI service that answers text questions as naturally as if it were a human, and if you ask it well, it will organize and give you the information you need. Therefore, you may feel that AI is much more intelligent than humans, that it knows everything and is all-powerful. But is this really true?
AI is not yet more intelligent than humans; it is being developed to be highly competent at specific tasks, but this is not the same as having the overall intelligence of a human.
Some people wonder if AI will take away their jobs. For example, AI is used to analyze large amounts of data, and then humans make judgments based on the results and numbers of the analysis. In many cases, it is more appropriate to think of AI as enabling intelligent labor on a scale that was not possible before, or as saving labor on existing human jobs.
You may be under the impression that AI is perfect and never makes mistakes, but in fact AI has its limitations and mistakes. There are two reasons. First, as is true of machine learning in general, AI's answers are "probabilities. You cannot always get 0% or 100% probability. Second, because AI makes predictions and decisions based on learned data, its accuracy depends on the quality and quantity of the learned data; AI does not know everything, and its ability is inevitably limited to new situations and unknown data. This is due to the fact that AI models exhibit very high accuracy for tasks within the scope of the training data (interpolation), while they do not perform well outside the scope of the absence of training data.
This can make it difficult to make sound decisions under sudden accidents or to make complex ethical decisions. Although highly capable AI may seem convenient and capable of anything, it is important for humans to place and utilize AI in the right places based on a good understanding of the problem to be solved and the characteristics of the AI. Only by making comprehensive judgments can humankind benefit from the dramatic progress made by AI.
AI is used in a wide range of fields, including AI that can play the famous game of Go (AlphaGo), AI-equipped chess software, image recognition AI used in face recognition and self-driving, voice recognition AI used in voice assistants such as Alexa and Siri, and natural language processing AI used in ChatGPT and others.
Natural Language Processing (NLP) is a technology that allows computers to process natural language used by people in their daily lives. The advent of deep learning has made possible advanced natural language processing that learns patterns from large amounts of data, and the Transformer is a major advancement in this field.
Deep learning is a type of machine learning, a learning method that uses a multi-layered neural network, similar to the structure and neural connections of the human brain, and is used not only for natural language processing but also for a variety of other applications. It can create approximate functions (models) that connect inputs and outputs in a very flexible manner. The resulting model outputs a list of optimal solutions to the input, with probabilities and scores.
Transformer is a model for natural language processing published in 2017. The model's ability to better capture context and meaning marked a major turning point in natural language processing, as the Transformer greatly improved the performance of machine translation, question answering, text generation, and sentence summarization.
ChatGPT, which has been a hot topic since the end of 2022, is a conversational, chat-type AI service developed by an organization called OpenAI. It is characterized by its ability to generate and respond to natural, human-like written expressions in response to user questions and input.
It is not accurate to say that ChatGPT is an AI, but rather, ChatGPT is the name of an AI service, GPT stands for "Generative Pre-trained Transformer," a language processing mechanism. It is a pre-trained generative transformer.
The reason why ChatGPT has become such a hot topic is because it is a "conversational service" that smoothly returns chats. The strength of ChatGPT is that it is now able to respond in natural sentences and words.
ChatGPT learns a lot of data through deep learning, finds patterns among them, and generates responses. Rather than outputting learned data as-is, ChatGPT can generate appropriate and natural responses to questions, which is a very superior form of natural language processing.
It can respond to general conversations and questions with a fairly high degree of accuracy, and its sentences are natural, but when it comes to specialized knowledge or complex questions, the responses tend to be inaccurate. Nevertheless, ChatGPT is useful enough to provide information and basic support, and can be conveniently used as a talker or assistant, so to speak.
After making such an impact, ChatGPT is surprisingly simple. It "keeps predicting the next word (token)," i.e., it keeps probabilistically predicting the next word in the text entered up to that point. To introduce naturalness, words other than the most probable may be used.
On the other hand, let's compare this to the situation in which you are speaking. You speak one word, choose the next natural word, think of the next natural word, and speak it again. What do you think of your own behavior in this way? It may be that he is specializing in natural speech rather than speaking with an image of what he wants to say, or that he is memorizing a pattern, a syntax, or something close to a grammar that strings many words together rather than thinking about logic. Conversely, language, which we thought was complex and mysterious, may be a relatively simple tool for expression.
Generative AI is AI that generates content based on pre-trained data, such as ChatGPT, which can generate text from a large amount of pre-trained data, and other AI that can generate images, designs, music, program code, etc.
FRONTEO's in-house developed AI engine "KIBIT" is also a natural language processing AI, but it is not a "generative" AI, but a "discovery" AI that searches for and finds necessary information from a huge amount of text data.
In today's world, where information is increasing every second, discarding and selecting information is a major challenge in any field and for any company. For example, a tax accountant investigating accounting fraud needs to find documents and interactions related to the fraud from a large volume of financial statements and related documents, while a lawyer in charge of litigation needs to find documents that can be used as evidence from a huge volume of related documents. Call center representatives are constantly searching for opinions to be utilized for product improvement or new development from various customer feedback such as inquiry history and questionnaires.
Instead of investing dozens of people to review all such data and documents, the AI engine "KIBIT" replicates the judgment and thinking of experts such as lawyers, tax accountants, and various business representatives by learning and selecting the necessary information.
KIBIT is a type of machine learning that uses a different algorithm than deep learning to analyze and learn text from very small amounts of teacher data at low power consumption and high speed on a laptop-level computer.
For example, documents related to the fraud and documents not related to the fraud are given to KIBIT as teacher data (source data). The documents can be a few pages of A4 paper or a single email, for example. At this point, KIBIT captures the atmosphere (subtleties) of the document as a whole and the surrounding area of the text of interest, based not only on words but also on the composition, number, and combination of letters used in the text, and extracts features and trends that quantitatively express the atmosphere, building an AI model for "discovery.
This is how KIBIT is resistant to word ambiguity and fluctuation, and can analyze as if it were reading the context. This means that even if the words used in the actual document to be analyzed are different from those in the first document, KIBIT will determine that the document is "related" if they are close in mood throughout the document.
There are various types of AI in practical use in the world, such as automated driving, voice and facial recognition systems, and chatbots. Even natural language processing AI for text and language is different between ChatGPT and KIBIT.
ChatGPT is a "generative AI" that is good at composing and summarizing sentences, while KIBIT is a "discovery AI" that searches for necessary information from a vast amount of text data by reproducing an expert's thoughts. KIBIT and ChatGP are also different. In other words, KIBIT and ChatGPT are not rivals competing for market share, but rather "AI friends" with different features.
KIBIT, which features highly accurate analysis of text data, was developed to support experts.
For example, for researchers in drug discovery, KIBIT suggests original "targets" for drug discovery from tens of thousands of papers. For a lawyer in charge of a lawsuit, KIBIT can present documents that can be used as evidence from a vast amount of relevant documents with high precision. For general companies and organizations, we analyze e-mails and SNS to align and give them useful information for business decisions and strategy formulation. Experts in medical, legal, intellectual property, finance, human resources ......, and various other fields are provided with the necessary and appropriate information to help them solve problems.
There is text data that can be used for KIBIT in any industry and any business. Any data that can be converted to text or text that can be analyzed by KIBIT is a target for KIBIT, including the familiar data we use in our work: emails and reports we exchange daily, phone conversations, interview and meeting records, and posts on social networking sites.
In doing so, it does not require large amounts of teacher data. With KIBIT, any text data can be used to assist experts in a variety of fields.
