1. Introduction

At present, the robotics industry is a key emerging technology industry that has attracted worldwide attention. SLAM is currently one of the most significant technologies in the process of realizing robot automation. SLAM is a methodology aimed at deriving the three-dimensional layout of unfamiliar surroundings alongside the sensor movements within that environment. This technology was originally proposed in robotics to realize autonomous control of robots. Among the SLAM technologies based on various sensors, visual-SLAM technology has attracted widespread attention because of its advantages of miniaturization and low cost [1]. Therefore, large-scale environmental SLAM based on visual information is an important technical field for mobile robot applications.

In early times of SLAM technology, the main focus was on the technology applied to a single robot. In contrast, multi-robot systems are more outstanding as the shorter execution time and higher reliability. Through multi-robot collaboration, using the collected visual information to jointly model [2], Multi-robot systems can be used for effective coordination and cooperation. Thereby completing tasks efficiently. At the same time, the exploration work in unknown environments such as underground, ruins, seabed, etc. [3] can use multi-robots to coordinate positioning and build environmental maps. These advantages in turn complement the advantages of visual SLAM. At the same time, the relevant algorithms of multi-robot SLAM are very similar to those of single-robot. This is because multi-robot SLAM was proposed during the gradual maturity of single-robot SLAM research, and researchers basically derived it based on single-robot SLAM technology. Expanded, the difference is that the realization of multi-robot SLAM will face more problems and require more technical support [4].

The key purpose of multi-robot SLAM is to finally draw a globally consistent map. Achieving this goal requires many aspects to be achieved together. Therefore, based on the application requirements of multi-robot collaborative visual SLAM, this paper summarizes the key scientific issues that exist in the current technology: how to allocate robot resources to implement SLAM-driven collaborative strategies, and how to integrate individual robots. The local map constructed to construct a global map to better represent the environment, and how to eliminate the back-end optimization of accumulated errors due to multiple sensor errors and so on. Then around the existing solutions related to each core problem, several benchmark methods for comparing different algorithms are provided, they are classified and their advantages and disadvantages are discussed, and the problems existing in the current multi-robot collaborative vision SLAM technology are analyzed. Finally, based on the above analysis, the research focus of multi-robot collaborative visual SLAM technology is discussed, and the development of this technology is prospected, so as to provide readers with a direction for the next research.

2. The main concept of multi-robot collaborative vision SLAM

2.1. Visual SLAM

The typical architecture for a visual SLAM system should include visual sensor capture, processing at front and back-end, closed-loop detection as well as map construction. The robot mainly uses the camera to acquire images, reads the information and performs a certain degree of preprocessing. The front-end acquires raw data from the visual sensor, performs preprocessing operations, and estimates the motion of the camera through changes in adjacent images. The backend needs to optimize the input results of the frontend, adjust the robot pose and map information, and obtain more accurate paths and maps. The current image information of the robot is associated with the historical map. If the similarity exceeds a certain value, the robot is considered to have passed the same place, so that a globally consistent state estimation can be obtained through back-end optimization, and problems such as robot position drift can be solved. Finally, according to the previous steps, build the map required by the task.

Vision sensors generally have the function of visual odometry, have sufficient stability and robustness, and are easy to implement. So far, many scholars have conducted in-depth research on visual SLAM and achieved significant results.

The ORB-SLAM method proposed by Mur-Artal et al. Since this method only retains some feature points in the image as key points, it is difficult to describe obstacles in the map. To this end, Mur-Artal et al. improved and designed ORB-SLAM2. ORB-SLAM2 will have certain improvements in accuracy and robustness.

LSD-SLAM and DTAM are featureless monocular SLAM methods based on direct methods, which mainly apply dense, pixel-by-pixel methods.

The Multi-State Constraint Kalman Filter (MSCKF) algorithm built by Mourikis et al. can achieve higher robustness in complex environments, and has higher accuracy and speed, and can run on embedded platforms with limited computing resources.

VINS-MONO is a powerful monocular visual-inertial state estimator, which is used in the fields of estimator initialization and fault recovery.

A summary of the above visual SLAM methods is shown in the table below.

Table 1. Summary of classic methods of visual SLAM.

2.2. Multi-robot SLAM algorithms

At present, most researchers main focus on the field of Filtering SLAM and Smoothing SLAM.

2.2.1. Filtering SLAM

EKF-SLAM: it is applied in the nonlinear system by using the Taylor’s first order expansion to simulate the liner system, however, the linearization error will accumulate over time.

Pros: If the quality of the features is good, EKF slam will be effective;

Cons: the mean and variance of the state variables must be stored. As the number of landmarks increases, the state variables increase in square series.

EIF-SLAM: it is similar to EKF, but using information matrix.

Pros: due to the additivity of information, the measurement update is carried out in a constant time, which is effective for multi-robot SLAM.

Cons: Owing to the sparsity of information matrix, the amount of calculation would increase which may reduce operational efficiency.

PF-SLAM: In none Gaussian system, EKF is not available, but Particle Filter (PF) can express the robots’ systematic status by posterior probability distribution which makes it effective for the nonlinear none Gaussian system.

Pros: It can better deal with the uncertainty of nonlinear and non-Gaussian distribution, so it has good robustness in dealing with noise and uncertainty. Also, it can be integrated with various sensors and is suitable for different environments and application scenarios.

Cons: only when the particles quantity is sufficient, the quality of the picture would be high.

2.2.2. Smoothing SLAM

Graph SLAM: It describes the topological relationship between the robot 's motion and the environment by constructing an optimization graph, and achieves accurate position estimation and map construction by optimizing the constraint relationship between nodes and edges.

Pros: By optimizing the relationship between nodes and edges, global consistent map construction and location estimation can be achieved. It can adapt to different scale and complexity of the environment, and can handle large-scale maps and a large number of sensor data.

Cons: the calculation requirement is high, and it is difficult to recover the covariance.

3. Task assignment

Multi-robot SLAM has many advantages, but at the same time, there are also some problems in multi-robot SLAM. For example, as the number of robots increases, they may interfere with each other. Therefore, how to allocate the resources of a group of robots and determine the execution order of each robot task, so optimizing the performance of the multi-robot system is very important. According to different task allocation strategies, the MRTA problem mainly has the following types of solutions: market-based methods, segmentation-based methods, and reinforcement learning-based methods.

3.1. Market-Based task allocation strategy

The market-based method requires each robot in the team to interact with task information. The cost is the expected moving distance to reach the target task point. The profit is taken by the robot moving to the area explored by the target point. Through market competition, the tasks of the robot team are assigned, so that teams of robot complete tasks while maximizing global profit.

Liwei Zhang et al. [5] introduced a multi-robot collaborative mapping method. Firstly, this method is used to quickly and completely extract the boundary points on the map. Secondly, the market-based method is used to complete the task assignment of the robot team. Finally, the efficiency is judged according to the time and distance travelled by the robot from start to finish.

3.2. Segmentation-based methods

The segmentation-based approach utilizes map segmentation techniques to divide the unexplored global map into different local regions, which are then assigned to individual robot members.

BENKRID et al. [6] published a study on a collaborative method based on map segmentation and optimizing energy consumption at the same time. By segmenting the map to be explored, the remaining energy of the multi-robot system and the expected energy consumption are used as a standard, considering Factors such as the energy consumption of stopping and turning during the robot's exploration process, and find the most suitable path for each robot, aiming to minimize energy consumption within the multi-robot system, thereby exploring as many unknown environmental regions as possible.

3.3. Methods based on reinforcement learning

The method based on reinforcement learning lets the multi-robot system consistently ventures into uncharted territories within the environment uses the evaluation returned by the system for training and learning, and obtains the optimal control strategy for multi-robots by obtaining the maximum cumulative reward value.

VISERA et al. [7] proposed a method using reinforcement learning that enables parameter sharing among individual robots. By extending the existing model for multi-robot information gathering tasks, they introduced a reward function conducive to robot collaboration. This model combines relevant prior information to facilitate task allocation among robot members.

Junyan Hu et al. [8] employed a reinforcement learning method to enable multi-robot teams to reduce redundant exploration regions. To address obstacles in unexplored environments, they presented an integrated collision avoidance algorithm that relies on deep reinforcement learning. This approach can improve the learning speed and performance of algorithms, and reduce the time and energy consumed by multi-robot systems to explore the environment.

To sum up, the research on task assignment technology of multi-robot cooperative visual SLAM has achieved a certain degree of breakthrough, and the representative research achievements are listed in the table below.

Table 2. Classification of multi-robot SLAM task allocation methods.

4. Map merging

In order to draw a global map which can depicts the environment with multi robots SLAM, it shows the significance to merge the local maps from individual robots. Therefore, methods and techniques for map merging play a crucial role in multi-robot SLAM systems. The following paragraphs will discuss several typical techniques for multi-robot SLAM map merging and related issues. Currently, the mainstream methods for map merging are primarily based on probability estimation of map particle positions and matching of map features. Filtering algorithms mainly assign weights to map particles and use probability filtering to resample particles to obtain an estimate of the map. Feature matching involves extracting and comparing features such as points and lines from existing maps to merge them, requiring a certain degree of map overlap.

4.1. Particle filtering algorithm

Particle filtering is a state estimation algorithm based on Monte Carlo methods, commonly used in SLAM for simultaneous localization and map construction. In map merging, particle filtering can be used to estimate the transformation relationship between different robots. By utilizing the sampling and resampling processes of particle filtering, correspondences between maps of different robots can be obtained, thereby achieving map merging. HG. Jo et al. proposed an optimized Rao-Blackwellized Particle Filter (RBPF) algorithm that combines map information with particle states and shares map information among particles, optimizing map merging accuracy while conserving memory [9].

4.2. Feature extraction algorithms

Hough Transform is a feature extraction technique used to identify features in objects, such as lines, squares, etc. A study of map merging technique that relies on the Hough Transform was proposed by S. Carpin et al. back in 2008. They convert maps into the Hough space first. Then the conversion links between local maps was determined by computing the correlations between Hough maps [10]. Meanwhile, S. Saeedi et al. performed Hough Transform on two sub-maps, extracted point and line features from the peaks, and matched them to achieve map merging, enhancing computational speed and real-time performance [11]. Scale-Invariant Feature Transform (SIFT) is a classical method in image processing. It was commonly used for image feature extraction and matching. K. Wang et al. utilized the SIFT feature recognition algorithm to abstract feature points from visual images and maps, and add descriptors. They then used the Iterative Closest Point (ICP) feature matching algorithm to identify shared areas between different robots. Based on these shared areas, they computed the transformation relationship between different robots, thus achieving map merging [12]. S. Hadian Jazi also improved the accuracy and robustness of map merging using the SIFT feature recognition algorithm [13].

4.3. Image overlap

Treating map merging as an image registration problem eliminates the strict requirement for feature extraction and analysis of the environment, making computations simpler and more real-time. A kind of algorithm was proposed by researchers back in 2006. It was used to measure the maximum overlap between two maps by utilizing image processing techniques such as rotation and translation [14]. L. Ma et al. treated the merging of maps with different resolutions as an image registration problem and introduced an iterative approach based on Mean Squared Error (MSE) and Random Sample Consensus (RANSAC) to merge non-overlapping areas of maps [15].

Table 3. Main methods of map fusion.

5. Back-end optimization

Visual SLAM (VSLAM) produces incorrect estimated positional trajectory results because the visual odometry inaccuracies build up over time. The cumulative inaccuracies will prevent the V-SLAM system from obtaining globally consistent positioning and orbit data. It will cause a more significant effect on the positioning and mapping outcomes. The error accumulation will be more severe in multi-robot SLAM. Consequently, a crucial component of multi-robot SLAM is the back-end optimization of visual multi-robot SLAM, which can reduce the global error and create a global unified map.

In the SLAM back-end optimization of multi-robots, it is currently divided into a back-end optimization algorithm based on filtering and a back-end optimization algorithm based on graph optimization. The core algorithms are also introduced above. Next, this article will briefly introduce the latest back-end optimization technology and its effects.

5.1. Algorithms based on filtering

A newest study develops an reconstructing and optimizing algorithm for 3 dimensional RGB-D based on dynamic SLAM [16]. The core of this algorithm is a filtering-based algorithm. Its main feature is the back-end optimization of visual SLAM in a dynamic environment, which is of certain significance for multi-robot outdoor work mapping. The author uses deep learning and multi-view geometry to segment the image, and then the feature points are re-extracted and re-matched. Meanwhile, due to the dynamic environment, the image may be segmented many times, resulting in a large amount of calculation and poor effect. However, this problem can be solved by using an optimized point cloud with a dynamic feature filter.

Another research in 2021 built a unique particle filter which is optimized in order to address the issue that particle diversity loss can result in particle impoverishment [17]. This is an optimization algorithm for PF-SLAM, which uses Opposition based High Dimensional optimization Algorithm (OHDA) to increase the particle set of PF. Compared with PSO-PF, the error value of OHDA-PF is about 30 times smaller. At the same time, when a PSO-PF cycle takes 5000ms, OHDA-PF only takes 400ms.

5.2. Algorithms based on graph optimization

A novel UWB range and graph optimization system was introduced in 2022 [18]. This system uses the idea of graph optimization to upgrade the accuracy and stability of the relative pose estimation of multi-robots. UWB tags are implanted on each robot to establish the link between UWB tag position and robot posture, reflecting the geometric relationship between each robot. A single estimation can start the judging process swiftly. To attain the goal, the sliding window estimate for precise relative attitude estimation and the rejection of outliers based on motion restrictions are applied. In the experiment, the time consumption of the single estimation of the improved algorithm is less than 0.1ms, and the relative position accuracy and the maximum detection distance under the condition of approximate equality are better than those of the algorithm based on the EKF.

In document [19], it provides an improved vision-based PG-SLAM graph optimization algorithm to reduce the large amount of computation caused by false positives when optimizing the graph. The principle is that the Siamese convolutional neural network (SCNN) compares two images and outputs existing overlapping information about the two parts of the environment they describe, which is verified by image-based cyclic filtering (ILF). Finally, the pose-based loop filter (PLF) uses the relative pose between them to verify again. After the experiment of semi-synthetic data, the Area Under Curve (AUC) after SCNN can reach 0.992. And in the use of PLF can be up to 300 frames per second, even with the ILF can be stabilized at 150 frames per second or more, that is, can reach 1ms.

5.3. Multi-robot collaborative visual SLAM back-end optimization summary

In summary, there are two main ways to decrease the accumulated error in multi-robot collaborative visual SLAM. The main characteristics, calculation accuracy and efficiency of these two ways are shown in Table 4.

Table 4. The main algorithm of robot collaborative visual SLAM back-end optimization.

6. Discussion

According to the discussion in Section 3, in the multi-robot task assignment problem, market-based methods can usually assign tasks to robots with reasonable performance because they rely on the decision-making mechanism of market auctions. The global task assignment is realized in a uncertain environment, but the spatial distribution of the robot has great randomness in different exploration scenarios; Segmentation-based methods can distribute robots as evenly as possible in the environment, avoiding mutual interference between robots. However, if the exploration space is too small for too many robots, it will affect the efficiency of task assignment, so the segmentation-based methods is more suitable for working in a large range of scenarios; Methods based on reinforcement learning rely less on external guidance information, do not need to establish precise mathematical models of the environment and tasks, and have faster responsiveness and higher adaptability, but training is difficult and time-consuming.

Based on the analysis in Section 4, classical RBPF algorithms can usually accurately complete the merging of sub-maps. However, the issue of sensor error accumulation is not easily addressed in this algorithm. At the same time, the inherent problems of high memory usage and computational complexity in filtering algorithms are particularly amplified when dealing with multi-robot exploration of large maps, requiring optimization. According to the timing and quantity of relevant literature, map fusion based on feature recognition algorithms is currently the mainstream trend in research. Typical feature recognition algorithms such as Hough Transform and SIFT have the advantage of high robustness. However, as maps become larger and more complex, the ability to handle complex graphics and real-time calculations needs improvement. With the rise of fields like deep learning and artificial intelligence, feature recognition algorithms have a broader outlook. Map fusion based on image overlap is mainly related to image registration and image blending techniques. Treating maps as simple images for overlap is simpler, with better performance in terms of computational load and real-time processing. However, this method requires a higher degree of overlap between maps, which is not conducive to the efficiency of multi-robot exploration. The aforementioned three categories of algorithms are not mutually exclusive in the field of map fusion; technology-fusion-type algorithms can complement each other to some extent. For example, feature point recognition can assist in improving the accuracy of image registration.

By analyzing the robot SLAM back-end optimization technology in the 3 years in section 5, we can get some conclusions. 1)The back-end optimization is now the core issue in improving the multi-robot SLAM. The iterative update of back-end optimization is developing rapidly, but it is undeniable that it is still in its infancy. 2)By comparing the characteristics of the back-end technology based on filtering or graph optimization, the back-end optimization based on filtering, whether EKF or PF, requires a lot of calculation to achieve high-precision requirements, which undoubtedly leads to the disadvantage of low computational efficiency. This disadvantage is particularly serious for multi-robot SLAM, because the slow speed of image update will have a negative impact on map fusion and the next trajectory of the robot, which reduces the advantage of multi-robot high efficiency. 3)The image refresh speed of the back-end optimization technology based on graph optimization is much higher than that of the related technology based on filtering. The optimized accuracy is slightly worse, but it will not significantly affect the final output map.

7. Conclusion

At present, in the unknown complex environment, the multi-robot task assignment method still has shortcomings, its spatial distribution is poor, and its adaptability to environmental changes is low. Under different criteria, the choice of task allocation strategy will be affected by various factors such as the specific application scenario, the complexity of the environment, and the performance of the multi-robot system. Therefore, it is of great research significance to comprehensively weigh various optimization indicators, allocate robot tasks reasonably according to different dynamic factors, and improve the efficiency of the entire robot team. Map fusion algorithms are diverse and each has its own advantages and disadvantages. Currently, map fusion algorithms based on feature recognition appear to be the more mainstream direction of development, with broader prospects. Its future development needs to focus on optimizing real-time performance and enhancing the ability to recognize complex graphical features. Back-end optimization technology based on graph optimization will have better research prospects. It will also be one of the valuable topics of multi-robot SLAM research in the future which focus on the effect of reducing error.

Authors Contribution

All the authors contributed equally and their names were listed in alphabetical order.