1. Introduction

In the field of artificial intelligence, an agent is a fundamental concept referring to a system capable of perceiving its environment and making decisions based on the perceived information to achieve specific objectives. Simply put, an agent can be likened to an employee in a company who receives assigned tasks and makes a series of decisions based on the provided information to accomplish these tasks. Multi-agent systems are computational systems composed of multiple interacting agents. Within these systems, agents make decisions by sensing the environment to achieve individual or collective goals. The collaborative and communicative nature of multi-agent systems enables them to handle complex, flexible, and scalable tasks effectively. They have been widely applied in areas such as traffic systems, environmental monitoring and protection, and economic market simulations.

Before the rapid advancement of large language model (LLM), traditional agents faced significant limitations in several aspects: limited natural language understanding and generation capabilities, restricted knowledge storage and reasoning abilities, insufficient adaptability and learning capacity, lack of multi-modal interaction abilities, weak generalization performance, inadequate interactivity, and a lack of memory and contextual understanding. These limitations severely hindered the development of agents [2-3]. In recent years, with the rapid development of LLM, LLM-based agents have developed rapidly due to their unique advantages in language understanding, text generation, planning reasoning, and knowledge integration.

LLM-based agents have significantly enhanced natural language processing, facilitating more efficient communication and interaction between agents and humans. Additionally, they have improved knowledge integration and storage, enabling access to and storage of vast amounts of knowledge, thereby exhibiting strong generalization performance across various tasks. With the capabilities of LLM, agents can interact smoothly with the environment, receive feedback, and continuously learn to adapt to new tasks, demonstrating self-evolution capabilities. Their reasoning, planning, and decision-making abilities have also significantly improved, making them adept at task decomposition, action prediction, and optimal decision-making [4].

Although LLM-based single agents excel in these aspects, they still face limitations in handling complex tasks, multi-modal data, long-term memory, computational resource demands, scalability, and coordination. These limitations have prompted researchers to explore LLM-based multi-agent systems, leveraging collective intelligence and specialized configurations and skills of multiple agents to overcome these challenges. Ongoing research has indeed shown that LLM-based multi-agent systems improve upon single-agent systems in various aspects:

Improvement in the accuracy of task execution: In multi-agent systems, complex problems can be decomposed into multiple sub-problems and assigned to agents with expertise in specific domains. This allows each agent to perform optimally within its domain. For example, in a courtroom scenario, agents can act as judges, jurors, plaintiffs, and defendants, each with specific tasks and responsibilities. This role allocation enhances task execution accuracy and overall system coordination and efficiency.

Enhancement of Robustness: By collaborating, multiple agents can better mitigate the impact of individual agent failures, thereby enhancing the system's robustness.

Increase in Adaptability: Multi-agent systems can dynamically reassign tasks based on environmental changes or task requirements, continually adapting to current objectives. This flexible response mechanism endows multi-agent systems with strong adaptability.

Boost in Efficiency and Innovation: Through collaboration and communication, multi-agent systems can allocate resources and tasks more efficiently, reducing unnecessary work and increasing overall productivity. Agent collaboration also fosters the exchange of individual insights, generating collective intelligence to explore more innovative solutions.

These advantages demonstrate the unique capabilities of LLM-based multi-agent systems in handling complex, dynamic, multi-domain tasks and highlight their promising development prospects [1], [5].

Given the current rapid development of LLM-based multi-agent systems, this survey aims to summarize the current state of research, analyze the specific roles of LLM in multi-agent collaboration, address current research challenges, and explore future development directions for LLM-based multi-agent systems

This paper is organized as follows. The second section of this paper will focus on the development history and current status of multi-agent systems; the third section will highlight the role of LLM in multi-agent systems; the fourth section will analyze current research progress, discuss existing problems, and potential future developments; finally, the paper will conclude with a summary.

2. Background

2.1. Development of Multi-Agent Systems

Multi-agent systems are a significant research direction within the field of artificial intelligence (AI), characterized by a diverse and interdisciplinary developmental history. The following is a detailed overview of the evolution of multi-agent systems.

The research on multi-agent systems originated in the 1970s, where it was explored as a branch of distributed artificial intelligence. During this phase, researchers primarily focused on designing agents capable of decision-making and actions specific to particular tasks.

In the late 1980s, with advancements in computer technology, research on multi-agent systems began to emphasize how to facilitate collaboration and communication among multiple agents to collectively accomplish complex tasks defined by the system. During this period, multi-agent systems found widespread applications in various fields, including intelligent robotics, business management, and traffic control.

By the 1990s, theories related to multi-agent systems had gradually matured. Researchers introduced several multi-agent reinforcement learning algorithms. For example, Littman proposed a multi-agent reinforcement learning framework based on Markov decision processes, laying the mathematical foundation for solving multi-agent reinforcement learning problems [6].

Entering the 21st century, breakthroughs in deep learning technology brought new opportunities for the development of multi-agent systems. Researchers began combining deep learning methods with traditional reinforcement learning algorithms, leading to the formation of numerous deep reinforcement learning algorithms, significantly advancing the research and application of single-agent reinforcement learning [6].

In recent years, multi-agent reinforcement learning (MARL) has become an emerging research hotspot. MARL integrates reinforcement learning with game theory and other techniques, enabling multiple agents to interact and make decisions in high-dimensional and dynamic real-world scenarios to accomplish more complex tasks [1].

Currently, research on multi-agent systems is progressing towards greater intelligence and automation. Researchers are exploring how to fully utilize the unique advantages of LLM to enhance agents' reasoning and planning capabilities, as well as how to solve complex real-world problems through multi-agent systems. With continuous technological advancements, LLM-based multi-agent systems are expected to demonstrate their unique potential and value in the near future [5].

2.2. Current Research on LLM-Based Multi-Agent Systems

At present, research on LLM-based multi-agent systems is experiencing rapid growth. These systems exhibit immense potential in solving complex problems through agent information reception and interaction. The following is a specific analysis of the current state of research on LLM-based multi-agent systems.

Multi-Agent Communication Frameworks: Researchers have proposed various multi-agent communication frameworks, leveraging LLM agents to engage in role-playing and interact with each other to adapt to diverse problem scenarios. For example, the "LLM Harmony" framework involves multiple LLM agents with different personalities participating in the communication process [7]. This approach offers a nuanced and effective method for addressing diverse problem contexts.

Complex Problem Solving and World Simulation: LLM-based multi-agent systems have made significant progress in both simulated environments and real-world tasks. For instance, the ChatDev project utilizes multiple agents within a multi-agent system to perform different roles, including CEO, Chief Product Officer, and Chief Technology Officer, thereby advancing various stages of software development [8].

Agent Configuration and Communication: In multi-agent systems, the configuration of agents and the communication mechanisms are critical. Agents are assigned specific roles based on their characteristics, behavior, and skills to collaborate towards achieving specific goals. The communication mechanisms mainly include cooperative, argumentative, and competitive paradigms. Additionally, communication structures can take various forms, such as hierarchical communication, decentralized communication, and centralized communication [5]. Depending on the environmental and task allocation needs, appropriate structures and communication paradigms can be set.

Modularization of Agent Systems: Researchers have also proposed modular solutions for agents to assist future researchers and enthusiasts in continuously developing more relevant agents based on existing templates. For example, ToolLLM provides a comprehensive data-building, model training, and evaluation template, facilitating the development of more functional agents [9].

3. The Role of LLM in Multi-Agent Frameworks

LLM play a crucial role in multi-agent collaboration due to their unique advantages. With the support of LLM, agents can efficiently collaborate and successfully execute tasks, achieving significant outcomes. This section delves into the key roles of LLM in multi-agent collaboration to further clarify the collaboration process and its importance in multi-agent systems.

3.1. Environment Construction and Dynamic Adjustment

In multi-agent systems, the construction and dynamic adjustment of the environment are vital, and LLM play a critical role in this process. LLM can build the required environment for multi-agent systems based on specific tasks and adapt to dynamic environmental changes. Currently, LLM can construct environments categorized into virtual environments, real physical environments, and non-specific environments [5]. Depending on the background and tasks, LLM can create suitable environments. In constructing virtual environments, LLM generate detailed task descriptions using their text generation capabilities, providing initial states and background for agents to collaborate effectively and execute tasks in the preset environment. In constructing real physical environments, LLM generate environment models using sensor data and prior knowledge, providing more accurate information for agents. In some cases, such as when multiple agents need to reach a consensus through debate, LLM do not set a specific environment but instead define the rules, goals, and constraints of the task, helping agents understand the task and interact effectively. Due to the variability and complexity of tasks, environments are not static. LLM can receive information on environmental changes and interact with agents to adapt dynamically, ensuring smooth multi-agent collaboration.

3.2. Specialization and Role Assignment

In the role assignment process, LLM can assign different roles and related information to agents based on task requirements and the agents' attributes and characteristics. The current role assignment methods mainly include predefined, model-generated, and data-derived roles [5]. In predefined role assignments, LLM designate fixed roles for agents based on their initial design and functional requirements. In model-generated role assignments, LLM can generate models with different expertise and capabilities tailored to specific tasks to meet the demands of specific work environments. In data-derived role assignments, LLM analyze historical data to identify agents' performance in past tasks and dynamically adjust role assignments to optimize the entire multi-agent collaboration process.

3.3. Collaborative Decision-Making

In multi-agent collaboration, the cooperation and decision-making among agents are crucial. This can be analyzed from three aspects: Firstly, in terms of communication paradigms, LLM support multiple communication paradigms, including competitive, cooperative, and debate-based paradigms, selecting the appropriate one based on the specific task needs to facilitate agent interactions. Secondly, regarding communication structures, the communication structure among agents varies according to the goals. Common communication structures include hierarchical communication, decentralized communication, centralized communication, and shared message pools. Selecting the appropriate communication structure based on task requirements significantly enhances collaboration efficiency. Lastly, concerning communication content, LLM-based multi-agent systems typically use text forms for communication. The natural language understanding capabilities of LLM enable efficient and smooth communication among agents [5],[10].

3.4. Action Execution

In multi-agent systems, upon receiving specific tasks, LLM first need to understand the task requirements and use their advanced language understanding capabilities to interpret these tasks and translate them into feasible action plans. LLM can break down action plans into a series of specific steps, facilitating internal system understanding and execution of tasks. Additionally, in dynamic environments, LLM can update action plans in real-time, ensuring that agents can effectively respond to environmental changes and accurately execute corresponding actions [10].

3.5. Evaluation and Feedback

In multi-agent systems, evaluation and feedback are important indicators of system performance and play a key role in system optimization. After agents execute a series of actions, LLM can evaluate their performance by analyzing the gap between action outcomes and expected goals, providing feedback on the effectiveness of the actions. Agents can learn and self-improve based on this feedback, continuously enhancing performance. Furthermore, LLM can conduct in-depth analyses of task results, identifying key factors for successful task completion or potential reasons for failure, thereby providing agents with successful experiences and warnings to avoid failures. This feedback mechanism helps optimize the overall efficiency of the multi-agent systems [11].

4. Current Issues and Future Development

LLM-based multi-agent systems have demonstrated excellent performance in handling complex tasks. However, several issues still constrain their further development during research and application. The following section provides a detailed definition and analysis of these issues, explores current methods to address these challenges, and proposes viable future solutions to further promote the development of multi-agent systems.

4.1. Multimodal Integration

The issue of multimodal integration involves effectively incorporating different modalities of data (such as text, audio, etc.) within the system to enhance its ability to handle complex scenarios. In multi-agent systems, multimodal integration enables agents to better understand their environment, thus improving task execution and collaboration, which is of significant importance. However, current multimodal integration faces certain limitations in areas such as data alignment, information loss, and fusion methods. These limitations lead to decreased collaboration efficiency, reduced decision quality, and weakened adaptability to the environment in multi-agent systems. To address multimodal integration issues, researchers have conducted various studies aimed at improving multimodal integration capabilities. Methods such as multimodal deep learning and Transformer models have contributed to enhancing multimodal integration [12]. Future research directions can focus on exploring advanced fusion methods to promote multimodal integration while maintaining information integrity and fusion suitability during the integration process.

4.2. Handling Hallucinations

In multi-agent systems, hallucinations refer to the phenomenon where the content generated by the model is inconsistent with real-world facts or user inputs. The occurrence of hallucinations may stem from data defects, issues during the training process, or incomplete reasoning strategies. This phenomenon raises concerns about the decision accuracy of multi-agent systems and significantly reduces collaboration efficiency among agents. To mitigate or eliminate the impact of hallucinations, several methods have been proposed, such as data optimization, adversarial training, and model improvements, aiming to reduce data-related hallucinations [13-14]. Future research can focus on improving data quality and enhancing model architectures to more effectively mitigate or handle hallucination phenomena.

4.3. Collective Intelligence Acquisition

Collective intelligence acquisition in multi-agent systems refers to the ability to surpass the capability of a single agent through collaboration and information sharing. If multiple agents cannot effectively collaborate to acquire collective intelligence, the trustworthiness and reliability of the entire system will be compromised, thereby losing its superiority. Current research, such as establishing knowledge-sharing mechanisms and consensus algorithms, aims to promote information exchange among multiple agents to acquire collective intelligence [15]. Future research can explore more effective methods to enhance the collaboration capabilities of multi-agents, ensuring that the collective intelligence obtained through collaboration aligns as closely as possible with the expected results of the actual tasks.

4.4. System Scalability

System scalability refers to the ability of a multi-agent system to maintain performance and efficiency as its scale and complexity increase. Currently, system scalability faces issues such as resource limitations, high communication overhead, and coordination difficulties, leading to limitations in the stability of multi-agent systems. Existing research efforts aim to promote system scalability through methods such as modular design and distributed architectures [6]. Future research can focus on exploring mechanisms for adaptive system scaling according to demand, to meet the needs of complex and changing environments.

5. Conclusion

This paper delves into the current state of research on multi-agent systems based on LLM, highlighting the critical role and potential of LLM in facilitating multi-agent collaboration. With the introduction of LLM, multi-agent systems can more effectively comprehend task requirements, foster agent generation, and enhance inter-agent collaboration, leading to more rational decision-making. However, despite the significant advantages LLM bring to multi-agent systems, current research and applications still face several challenges, including environment handling, multimodal integration, and scalability. Therefore, continuous in-depth research and exploration are necessary. As artificial intelligence technology continues to advance and make breakthroughs, LLM-based multi-agent systems are expected to better address complex problems that perplex humans, demonstrate their unique value in more fields, and provide more powerful and flexible tools to tackle real-world complex challenges in the future.