1. Introduction

Chatbots are computer programs designed to simulate human communication through text or voice interaction. They are a type of artificial intelligence technology that uses natural language processing (NLP) and machine learning algorithms to understand and respond to user queries. Chatbots can be used in a variety of applications, such as customer service, healthcare, and e-commerce, to provide instant responses and personalized experiences. Based on the wide references to chatbots in different industries, people are gradually accepting the increasing involvement of chatbots in their lives. With the iterative updating of computer-related technologies such as machine learning, the intelligence of chatbots is also increasing, and their potential market prospects are also being explored. Therefore, it is necessary to understand the development history and iterations of chatbots from the 1960s to the present.

Through a literature review and theoretical analysis, this paper explores the development and future of chatbots with the theme of the history of chatbots. By summarizing the history of chatbots, it is hoped that readers will also gain a preliminary understanding of the development process of chatbots so that they can better understand this new tool.

2. History of chatbots

2.1. The beginning of chatbots - ELIZA

The history of chatbots dates back to the 1960s, when Joseph Weizenbaum developed the first chatbot, ELIZA. ELIZA is a rule-based program that uses pattern recognition and substitution to simulate human-like dialogue. Figure 1 shows ELIZA as a chatbot assisting a psychotherapist in talking to a patient. It analyzes user input and gives preprogrammed responses based on keywords and phrases. ELIZA is designed to mimic a psychotherapist and is used to explore human-computer interaction and natural language processing.

Despite its limitations, ELIZA is a breakthrough in the field of artificial intelligence research, paving the way for the development of future chatbots. It demonstrated the potential of chatbots to communicate with humans in natural language, and sparked interest in the field of artificial intelligence. Weizenbaum also found that when people talk to chatbots, they are more willing to confide their truest and deepest thoughts. But he was only willing to make chatbots be an auxiliary device or tool [1].

Figure 1. ELIZA chat with a patient.

2.2. Chatbots in the 20th and 21st century - from parry to siri

2.2.1. PARRY. PARRY is a chatbot developed by Colby in the mid-1970s. The chatbot was designed to mimic paranoid patients for the purpose of studying human behavior.PARRY was one of the first chatbots ever developed, and it paved the way for the development of many other chatbots in use today [2].

The PARRY chatbot was programmed to mimic paranoid behavior in response to user input. The chatbot was designed to be sensitive to criticism, reacting defensively and accusing the user of being part of a conspiracy against it.PARRY’s responses were based on a set of rules programmed into the system, and the chatbot was also capable of generating new responses through a technique known as “elaboration”. Although PARRY was never intended to be used for commercial purposes, its development laid the groundwork for the development of the modern chatbot. Today, chatbots are used in a variety of industries, including customer service, healthcare, and finance. Chatbots are also becoming sophisticated due to advances in artificial intelligence and machine learning. Despite the age of its invention and utilization, PARRY remains an important milestone in the development of AI and chatbots [2].

2.2.2. Smarterchild. Smarterchild is an AI-powered chatbot launched by ActiveBuddy Inc. in 2001. It is designed as an interactive digital assistant that can communicate with users through various messaging platforms, such as AOL Instant Messenger, Windows Live Messenger, and Yahoo Messenger. SmarterChild quickly became popular due to its ability to provide fast and accurate responses to user queries. It can answer questions about news, weather, sports, and even play games. Chatbots are designed to learn and evolve as they interact with users. It can recognize user preferences and adjust its responses accordingly. For example, if a user asks SmarterChild for the latest news updates, the chatbot will remember the user’s preferred news sources and provide updates from only those sources in the future. SmarterChild is also programmed to interact with users in a conversational tone, making the experience more natural and enjoyable. It understands natural language and responds appropriately, making it feel like you’re chatting with a real person. Overall, SmarterChild is a breakthrough innovation in the chatbot space. It paves the way for the future of AI-powered digital assistants and inspires other companies to develop similar products. Although SmarterChild was discontinued in 2008, it remains a legend in the history of chatbots and has had a profound impact on the world of digital communications [3].

2.2.3. Siri. In 2011, Apple introduced Siri, an AI-based chatbot that represents a significant breakthrough in the field of AI technology. By leveraging natural language processing and machine learning techniques, Siri is able to understand user requests and provide personalized responses. This chatbot represents a new era in the field of virtual assistants, setting the standard for future developments in this area. Siri is capable of performing a variety of tasks, such as setting reminders, sending messages, and making phone calls, making it an invaluable tool for many users. Overall, Siri represents a game-changer in the field of AI technology and has had a significant impact on the way we interact with digital devices.

2.2.4. Chatgpt. ChatGPT is a state-of-the-art chatbot developed using advanced machine learning techniques. It aims to simulate human-like conversations and provide informative and engaging real-time responses. ChatGPT was developed by using several different models, including ChatGPT1, ChatGPT2, ChatGPT3, ChatGPT3.5, and ChatGPT4 (Figure 2) [4].

Figure 2. Summary of major dataset sizes. in gigabytes. published data are in bold. determined data are in italics. only the original training dataset size.

The development of ChatGPT started with the creation of ChatGPT1 to provide users with a basic chatbot experience. The model is based on a simple language model trained on a large text dataset. The first generation of GPT model, when it was born in June 2018, was a powerful language understanding model. It is not difficult to judge the semantics and relationship between two sentences, classify text data, question and answer and common sense reasoning, but it is not a very good conversational AI model, and the training parameters are much lower than the subsequent models. As users interact with ChatGPT1, the model is able to learn and improve their responses, which led to the development of ChatGPT2 [5].

GPT-2 In February 2019, OpenAI launched GPT-2, which evolved from GPT-1, but the main change is the use of more parameters and data sets, with the number of parameters reaching 1.5 billion (GPT-1 only 117 million), the learning objective was changed to “no task-specific training”. This proves that greatly increased parameters and data can make GPT-2 a higher level than GPT-1. Although the performance on some tasks is not better than random, it has certain advantages and breakthroughs in generating short texts and making up stories. ChatGPT2 represents a major improvement over its predecessor, capable of generating longer and more complex responses better suited for real-world conversations. The model was trained on a larger dataset of text, which allowed it to develop a deeper understanding of language and context.

In 2020, GPT-3 also simply and rudely piled up more computing resources with money, continuing the previous GPT one-way language model training method, but the model has increased to 17.5 billion parameters. GPT-3 has made a major breakthrough in the field of natural language processing, becoming the largest and most powerful natural language generation model at that time. From machine translation to article summary output, it has excellent performance. However, due to the severity of the epidemic in 2020, people have not paid enough attention to breakthroughs in the field of artificial intelligence. In addition, compared with ChatGPT, GPT-3 cannot conduct natural conversations and can only handle one-way tasks, so only a few developers are interested. It was not until the end of November 2022 that OpenAI released the update of “GPT-3.5”, which focuses on the dialogue mode, and can even admit mistakes and reject improper requests-this is the model behind ChatGPT, which is closer to the characteristics of human dialogue and thinking. s concern. The development of ChatGPT3 marks an important milestone in the development of chatbots. The model was trained on unprecedented amounts of data, enabling it to generate responses nearly indistinguishable from humans. ChatGPT3 is able to understand and respond to a wide range of topics, making it one of the most advanced chatbots on the market [6].

ChatGPT3.5 and ChatGPT4 are the latest iterations of the ChatGPT model with more advanced features. These models have been trained on larger datasets, enabling them to generate more nuanced and complex responses. ChatGPT4, in particular, promises to be a major breakthrough in the field of chatbots, capable of generating real conversational responses comparable to humans [7].

On March 14, 2023, OpenAI released GPT-4 again. This time the accuracy rate has increased by 40%, mainly focusing on information sorting and searching on the Internet. , can also support visual input, image recognition, and know how to “tell stories by looking at pictures”! However, GPT-4 didn’t cost a fortune and didn’t pile on training parameters. Instead, the focus of R&D will be on improving the ability to use existing data [8].

In summary, the development of ChatGPT is an important milestone in the development of chatbots. From the basic features of ChatGPT1 to the advanced conversational features of ChatGPT4, these models demonstrate the potential of machine learning to create truly intelligent and engaging chatbots. As technology continues to evolve, we can expect to see more sophisticated chatbots in the future that have the potential to change the way we interact with technology.

3. AI and machine learning in chatbots

Advancements in AI and machine learning have revolutionized the chatbot industry, and advances in artificial intelligence and machine learning also provide more possibilities and development directions for the development of chatbots. AI-based chatbots can understand and respond to complex user inquiries, learn from user interactions, and improve their responses over time. Machine learning algorithms enable chatbots to analyze vast amounts of data and identify patterns to provide personalized experiences

The use of artificial intelligence and machine learning in chatbots has been increasing in recent years. Chatbots are computer programs designed to simulate human conversation through text or voice interaction. Businesses are using them to improve customer service, increase engagement, and reduce costs. With the help of machine learning, chatbots have become smarter and more efficient at understanding and responding to customer queries.

3.1. Applications and challenge

3.1.1. In customer service. Machine learning chatbots use natural language processing (NLP) algorithms to understand the context and intent of customer queries. They can analyze large amounts of data and learn from past interactions to provide more accurate and relevant responses. This enables businesses to provide personalized support to customers at scale. Machine learning chatbots can also be trained to recognize patterns in customer behavior and provide proactive support.

Machine learning chatbots can also be trained to recognize customer behavior patterns and provide proactive support, which makes chatbots smarter and more efficient at understanding and responding to customer queries. Based on this characteristic, chatbots are mostly used in customer service, sales and marketing to increase engagement and reduce costs. Therefore, the main future direction for chatbots is personalized interaction. However, ensuring that chatbots can handle complex queries and maintain a human-like tone remains a challenge.

3.1.2. In marketing. Chatbots are used in sales and marketing to increase engagement and conversion rates. Chatbots can be programmed to provide product recommendations, answer frequently asked questions and guide customers through the sales process. They can also be used to gather customer feedback and insights to improve products and services.

Another challenge is to ensure that chatbots can maintain a human-like tone and provide a personalized experience to customers. The rise of artificial intelligence and machine learning in chatbots is changing the way organizations interact with their customers [9].

3.2. Advantages and limitations

3.2.1. Advantages. One of the main advantages of machine learning chatbots is their ability to run 24 hours for day and 7 days for week without interruption. They can handle a large number of queries simultaneously, reducing the need for human intervention. This helps businesses reduce costs and improve efficiency. Machine learning chatbots can also integrate with other systems like CRM and marketing automation tools to provide a seamless customer experience [10].

3.2.2. Challenge in application. The use of machine learning chatbots also presents some challenges. One of the main challenges is ensuring that chatbots can handle complex queries and provide accurate responses. This requires continuous training and development of NLP algorithms for chatbots.

Another challenge is ensuring that chatbots can maintain a human-like tone and provide customers with a personalized experience. In conclusion, the rise of artificial intelligence and machine learning in chatbots is changing the way businesses interact with customers.

4. Conclusion

The future of chatbots lies in natural language processing and personalized interactions. Natural language processing enables chatbots to understand and respond to user input in a more human-like manner. It can also help chatbots to recognize emotions and sentiment to provide more empathetic responses.

Personalized interactions are essential for chatbots to provide a personalized experience to users. AI-based chatbots can analyze user data, such as search history and social media activity, to provide personalized recommendations and responses. Personalized interactions can also increase user engagement and satisfaction.

As chatbots continue to evolve, they will become an integral part of our daily lives, providing personalized assistance and support. Chatbots have the potential to transform various industries, such as healthcare, e-commerce, and customer service. With the advancements in AI and machine learning, the possibilities for chatbots are endless.