1. Introduction

In recent decades, the development of technology has greatly improved human life by reducing human force in daily work. Robots, as a kind of technology with varies designs, have infinite potential serving in multiple areas. To enhance robots' adaptability to human work environments, scientists conduct research by observing the natural world. Natural selection has provided many inspirations for human innovation. Insects significantly inspire the design of robots, particularly through their locomotion patterns, leg structure, stability, and sensory adaptations. That’s how hexapod robots are designed. Hexapod robots’ designs often mimic insect gaits and can switch between different gaits to adapt to various terrains. The six-legged design offers significant advantages in stability, adaptability to diverse environments, and fault tolerance compared to other widely used robots. Hexapod robots have quickly garnered attention in both academic and industrial circles due to their unique design and capabilities.

The advantages allow robots to be used in multiple fields. In search-and-rescue missions, their ability to navigate different terrains makes them invaluable for locating survivors in disaster-stricken areas. In agricultural settings, hexapod robots can monitor crops, assess soil conditions, and even perform planting or harvesting tasks. Their design enables them to navigate rocky and unpredictable terrains, making them perfect for collecting data during planetary explorations and space missions. However, one aspect limiting the improvement of hexapod robots is developing practical control methods. A well-designed algorithm for gait planning is required to control this complex system.

Gait planning, which refers to the process of determining the sequence and timing of leg movements to produce desired locomotion patterns, is of great importance in the algorithm design. An effective gait planning ensures that the robot can navigate its environment efficiently. This can help the hexapod robot maintain stability and adapt to diverse environments. Hexapod robots utilize various common gaits, each offering distinct benefits in terms of speed, stability, and energy efficiency based on the conditions in which they operate. The most well-known gaits are the tripod gait, wave gait, and ripple gait.

Fixed pattern gait planning can lead to limitations when the robot encounters unexpected situations or accidents. To improve robustness and reliability, more advanced gait planning methods are developed. These algorithms go beyond these basic patterns by integrating real-time sensory feedback and adaptive algorithms. This allows the robot to respond dynamically to sudden disturbances. Techniques such as reinforcement learning, genetic algorithms, and neural networks are increasingly being employed to enhance the autonomy and performance of hexapod robots. This paper will focus on summarizing recent research in the development of gait planning algorithms. Tradeoffs between different designs will be mentioned.

2. Terrain-adaptive gait planning method for hexapod bionic robots

This section examined the advantages and disadvantages of various gait planning strategies. Firstly, fixed pattern gait planning is frequently employed. The best-known gaits, such as the tripod gait, wave gait, and ripple gait, are straightforward to implement but do not adapt well to different environments. When encountering sudden disturbances, like going on a ramp or meeting an impulse, the robot cannot maintain a smooth movement or is even unable to reach the desired speed. This is dangerous for real-world applications. Thus, advanced gait planning strategies that could adapt to complicated environments should be applied. This paper will be mostly analyzing two commonly used controlling algorithms: the central pattern generator, and the reinforcement learning. After that, the tradeoff of using the two strategies will be emphasized. In the end, the paper will talk about other useful strategies and suggestions for developing an advanced gait planning algorithm.

2.1. Algorithm based on Central Pattern Generators (CPG)

Appling hierarchical networks in motor pattern generation in hexapod robots and insect movement offer valuable insights for understanding and controlling both types of systems. Barrio et al. examine a Central Pattern Generator (CPG) model for insect movement, consisting of six coupled neurons that develop movement patterns [1]. They offered an in-depth examination of the potential gaits produced by the CPG using various recently developed numerical methods designed for studying small networks. This allows analysis of the complete model without simplifications.

They combine the analysis of individual neurons using multiple three-dimensional parameter spaces to offer a comprehensive overview of the neuronal dynamics at play. Additionally, they utilize lateral phase lag plots to illustrate the convergence and transitions towards specific patterns and apply a quasi-Monte Carlo sweeping technique to depict various pattern trajectories within the parametric phase space. From their test data of the experiment, most of the observed patterns follow routes that lead to the stable tripod gait.

The study uncovers the bifurcations that initiate and eliminate different pathways and demonstrates that for high values of certain parameters, only the tripod gait persists. Considering that most routes result in the tripod gait and the rest disappear due to certain bifurcations, the researchers developed an algorithm. This algorithm is used to locate the limit of the tripod gait in the parameter space. The algorithm is used to locate the limit of the tripod gait in the parameter space, indicating the presence of a substantial three-dimensional parameter space where the tripod pattern is prevalent and highly dominant in scenarios involving fast-moving insects. The mathematical analysis of the insect CPG also explains why the tripod gait is the gait of movement to run fast in generic insect movement [1].

The research about potential gait produced by the CPG constructed the foundations of gait control system design. Feedback controlling system is often used to help CPG adapt to various environments. Zhong et al. introduce a biomimetic approach inspired by central pattern generators (CPG) for controlling locomotion and planning the gait of an innovative hexapod robot, which features legs that can be distributed freely in a radial arrangement around its body [2]. The CPG model integrates biological neural oscillation principles with mechanical control strategies, offering a novel approach to locomotion control for legged robots. The combination of the CPG network and feedback mechanism enables the robot to adapt in real time, ensuring stable and smooth locomotion even on rugged terrain.

A CPG network consisting of six units based on Matsuoka's neural oscillator is utilized to generate oscillatory signals. The phase signals produced by the CPG network are converted into tip trajectory signals for the six legs using an adjustment function, and the joint angles are calculated through the robot's inverse kinematics. Additionally, the system uses processed feedback information to fine-tune the parameters that determine robot movement, allowing for easy application and adjustment.

To verify the effectiveness of the developed controlling method, experiments are performed for testing. Both virtual and real experiments demonstrated that the proposed control method enables the hexapod robot to walk steadily on rugged terrain. The testing results show that the controlling method enables a hexapod robot to move on rugged terrain with stability. However, this control method has limitations in enabling the robot to walk on more complex terrain in real time, which remains an area for further research. Despite the limitations in handling more complex terrain in real time, the study demonstrates significant progress. In summary, this research demonstrates a successful application of a biomimetic control method that improves the adaptability and stability of hexapod robots in difficult environments [2].

Similar work that combines CPG with feedback controlling system is done to improve terrain adaptation. This time, the circumstance for ramp climbing is added during the experiment. Wang et al. talk about gait planning of a hexapod robot using a Central Pattern Generator with feedback control [3]. The algorithm aims to achieve stable omnidirectional motion, particularly during transitions from flat surfaces to slopes and while climbing slopes. The study begins by addressing the landing point problem during transitions, proposing a new stability criterion. It defines the connection between the ankle angle of the supporting leg and the body's pitch. Then it plans the transition gait using CPG bottom feedback. This approach ensures that the hexapod robot can maintain stability and uniform foot force distribution during the transition from flat to inclined surfaces.

For slope motion, the relationship between the supporting knee joint angle and the pitch angle of the hexapod is analyzed. The findings lead to the planning of a slope gait using CPG middle-level feedback. The study presents a method for calculating the stability margin using vector products, which helps justify the robot's motion stability. This method guarantees that the centroid projection point remains within the supporting domain, thereby maintaining the robot's balance.

The research includes extensive simulations and physical hardware experiments using a MATLAB/ADAMS co-simulation platform. The hexapod robot successfully transitioned from flat to a 12° slope and climbed a 16° slope, demonstrating the effectiveness of the proposed gaits. The robot maintains stability and uniform foot force distribution throughout these motions. The results validate the transition gait based on Hopf bottom feedback, and the slope gait based on Hopf mid-layer feedback, showing improved stability and adaptability in unstructured environments.

In conclusion, the study presents a robust approach to gait planning for hexapod robots, enhancing their performance on varied terrains. Future research will focus on the link between yaw angle, roll angle, and stability, incorporating attitude feedback to improve the stability of the hexapod robot’s transitional movements. The proposed methods and stability criteria effectively improve the robot's ability to detect and adapt to complex environments [3].

2.2. Algorithm based on Reinforcement Learning (RL)

Liu et al. present a study on fault-tolerant tripod gait (F-TT) planning and verification for a hexapod robot, focusing on maintaining stability and locomotion when one leg fails [4]. The research looks into the crucial issue of component resilience in robots, with a particular focus on their performance in high-risk scenarios such as earthquake rescue operations. The hexapod robot, leveraging its redundant degrees of freedom, demonstrates remarkable adaptability by maintaining functionality even in the event of a single-leg failure. The study uses the Denavit–Hartenberg (D-H) method to establish a kinematic model and analyze the Jacobian matrix, proving that the robot can be controlled with three legs supported. A phase sequence planning method based on stability margin is developed, and a trajectory for the center of gravity (COG) and foot is studied. To enhance stability, the paper proposes an adjustment to the coxa initialization angle, which is calculated utilizing the Monte Carlo method (Figure 1).

Figure 1: Six kinds of support triangles [4]

Simulations and experiments were conducted to verify the effectiveness of the proposed F-TT gait under leg failure conditions. The results show that the hexapod can maintain stable and smooth locomotion with one leg fault, thanks to the planned gait and stability improvement measures. The study concludes that the F-TT gait effectively addresses the forward motion problem of the hexapod under single-leg failure, though future research will focus on uneven terrain and dynamic force control. This groundbreaking research significantly contributes to the development of more reliable and adaptable hexapod robots, substantially enhancing their performance in complex and unstructured environments [4].

To have a better performance, free gait planning is often used to improve flexibility. Liu and Xu address the challenge of planning the free gait of hexapod robots in complex terrains [5]. The authors propose a discrete gait model that discretizes the single footstep distance and uses the longitudinal stability margin to analyze robot stability. The gait planning problem is transformed into an optimization problem of position states during each oscillation period.

The paper introduces a novel free gait planning method based on an improved Deep Q-learning (DQN) algorithm. This innovative approach optimizes the experience replay mechanism, effectively reducing low-value iterations and enhancing convergence speed. The improved DQN algorithm uses prioritized experience replay to focus on more significant experiences, thus improving learning efficiency. The algorithm's performance is validated through simulations using V-rep and Python, where a hexapod robot navigates a rugged terrain with a 20° slope (Figure 2).

Figure 2: Foot drop point model [5]

The results show that the improved DQN algorithm achieves faster convergence and a higher average stability margin compared to the traditional DQN algorithm. Specifically, the improved DQN converges after 280 oscillation cycles with an average stability margin of 9.31 cm, while the traditional DQN converges after 480 cycles with a margin of 9.26 cm. The study concludes that the innovative improved DQN algorithm not only effectively plans stable and adaptive free gaits for hexapod robots, but also successfully mimics biological gait behaviors, thus ensuring remarkably stable movement in even the most challenging environments. This method leverages deep reinforcement learning and prioritized experience replay to optimize gait planning, demonstrating significant improvements over traditional methods.

Another article also provides practical methods for designing a free gait planning algorithm using reinforcement learning [5]. Wang et al. use free gait planning and deep reinforcement learning (DRL) to develop a motion planning method for hexapod robots [6]. The proposed method, termed HFG-DRL, addresses the challenges of legged locomotion in unstructured environments by decomposing the complex task into path planning in discrete state space and gait planning in continuous state space. The approach involves three key components: a path planner (PP) using a soft deep Q-network to obtain global prior path information, a free gait planner (FGP) to determine the gait sequence, and a deep reinforcement learning algorithm to generate the optimal center-of-mass sequence.

The paper presents the benefits of hexapod robots, particularly their superior stability and ability to traverse complex terrains, compared to wheeled and tracked robots. It also points out that conventional multicontact motion planning approaches tend to concentrate on optimizing each step individually, which can result in inefficiencies. The hierarchical approach of HFG-DRL improves planning efficiency and adaptability in dynamic environments. Through rigorous testing in a diverse array of simulated and real-world environments, from rugged terrains to urban obstacles, the experimental results convincingly demonstrate both the feasibility and remarkable effectiveness of the proposed HFG-DRL method.

The HFG-DRL method outperforms traditional methods in terms of average episode rewards, steps, and success rates. The paper also includes a comparison with other DRL-based methods and traditional path planning algorithms, showing the superiority of HFG-DRL in complex, unstructured terrains. In their compelling conclusion, the authors emphasize the far-reaching practical applications of the HFG-DRL method, from revolutionizing rescue operations to opening new frontiers in planetary exploration. They also outline an exciting future research direction: enhancing the algorithm's resilience in scenarios where the robot may face leg malfunctions, further expanding its potential applications in the most extreme and unpredictable environments [6].

2.3. Others

This section provides gait planning methods other than CPG and RL, but still provides valuable ideas to improve the performance of terrain adaptation.

Xue et al. developed a gait planning and control method based on velocity vectors. Previous research implemented an event-triggered active disturbance rejection controller (ADRC) technique for DC motor position tracking. This approach aimed to improve the control accuracy of the physical servo system in the presence of external disturbances and uncertainties [7]. Additionally, the diagonal gait sequence for the hexapod robot was created to allow for diverse walking patterns. A comprehensive kinematics model of the hexapod robot is applied, along with a detailed description of its standard gait sequence, laying a solid foundation for the subsequent analysis. The connection between the velocity vector and the center of rotation is discussed. Building on this, an innovative algorithmic procedure for diagonal and turning gaits, which is fundamentally based on the velocity vector concept. The simulation results indicate that the robot body can maintain turning with various radii during the diagonal gait, validating the effectiveness of the algorithm [7].

From the result CPG, tripod gait is of most efficient. Guo et al. presented a tripod gait planning design to show its usability [8]. The research presents the design and gait planning of a hexapod robot tailored for lunar exploration, addressing challenges posed by the Moon's loose, adhesive, and corrosive soil. The robot features hollow legs made from anti-corrosive aerospace materials to reduce soil adhesion and overall weight. An "inverted gait" motion mode is introduced to enhance movement speed and stability compared to traditional tripod gaits. Drawing inspiration from nature's own six-legged marvels, the robot's design cleverly mimics hexapod insects. Its innovative elliptical leg layout serves a dual purpose: it elegantly prevents potential leg collisions while ensuring an expansive swing space, thus maximizing the robot's agility on the lunar surface. The legs are structured with hip, knee, and ankle joints, allowing for flexible and stable movement on the Moon's unstructured terrain. The robot's drive system uses an STM32F407 control board and RDS3115 dual-axis digital servos for precise control.

Kinematic analysis using the D-H method and simulation in ADAMS2019 software validates the robot's design. The inverted gait mode, where legs swing backward initially, increases the effective step length and reduces joint vibrations, resulting in a 29.53% increase in movement speed and improved stability. Experimental tests on a prototype confirm the robot's ability to stand and walk on simulated lunar soil, with the hollow leg design effectively reducing soil adhesion. The spherical foot design prevents sinking, ensuring stable movement. In conclusion, this groundbreaking study demonstrates that the innovative hexapod robot design, coupled with its novel inverted gait mode, significantly enhances lunar exploration capabilities. The robot offers markedly faster and more stable movement, along with superior adaptability to the Moon's notoriously challenging terrain. Future research will explore other gait modes and further optimize the robot's mechanical structure for even more complex lunar environments [8].

3. Conclusion

In the field of gait planning for hexapod robots, there is a critical tradeoff between using Reinforcement Learning (RL) and Central Pattern Generators (CPGs). RL approaches allow hexapod robots to learn optimal gaits through trial and error. This enables them to adapt dynamically to complex and changing environments. This flexibility, however, comes at the cost of requiring significant computational resources and extensive training time. Conversely, CPGs leverage biological principles to produce rhythmic and stable movement patterns through simpler, oscillatory control mechanisms, offering robust and energy-efficient gait generation. While CPGs provide quick and reliable solutions with minimal computation, they lack the adaptability and capacity for optimization under novel conditions that RL can offer. Balancing these methodologies involves harnessing the adaptive strengths of RL while benefiting from the stability and efficiency of CPGs, potentially leading to hybrid approaches that capitalize on the best of both worlds.

Future work will focus on developing hybrid approaches that effectively combine the strengths of both methods. One promising direction is the integration of RL with CPGs, where RL can be used to adapt the parameters of CPG-generated movements, achieving adaptability and stability. Before combining the two algorithms, improvements could be made to them separately. Advances in computational power and more efficient algorithms will help reduce training times associated with RL. Moreover, using advanced sensor technologies will enhance real-time feedback, allowing for quicker and more precise adjustments in gait patterns. Another direction could be to evaluate the effectiveness of both algorithms under different conditions, weighing their performance to determine which is more suitable in various scenarios. Input both controlling methods into the robots so that they can switch between CPG-driven and RL-driven modes based on environmental complexity and task requirements.