1. Introduction
People have been considering imitating real-world action in the virtual world for a long time. Starting in 1915, the rotoscope skill invented by Fleischer was considered the beginning of the motion capture idea. Artists utilize their painting skills to depict what they have seen in the real world and to re-produce it to their anime characters. With the rapid development of computer hardware and software technology, people are demanding to make virtual elements as close to the real world as possible. People are increasingly improving animation production and are constantly trying to simulate the real world on computers. Among them, motion capture technology is one of the bridge technologies to build real and virtual worlds. With the practical application of motion capture technology, many companies have also launched motion capture-related equipment, such as Polhemus, MotionAnalysis, and so on. Motion capture technology is widely used in various animations, films, game productions, virtual reality-related industries, and so on.
This paper introduces the key idea behind motion capture technologies and today's mainstream major motion capture techniques including mechanical, acoustic, electromagnetic, optical, and inertia sensor-based motion capture methods with their advantages and disadvantages and several applications. This paper helps readers quickly become familiar with the field, summarizing the main research directions in the fields with their constraints and provides insights for future researchers to find their path.
2. Analysis of existing motion capture technologies
2.1. The principle of motion capture technology
The essence of motion capture technology is to measure, track and record the motion trajectories of objects in three-dimensional space through the combination of hardware and software, and then transfer the data to the computer for high-speed processing. The motion is restored and rendered in the computer model so that the virtual model in the computer can be well combined with the motion data in the real world so that the model can move.
To capture all the data in the real world comprehensively and accurately, the general motion capture device consists of sensors, signal capture devices, data transmission devices, and data processing devices [1].
(1) Sensor: A device used to track the specific location of an object. The position information of moving objects will be provided, and the more the number of sensor settings, the more detailed information will be captured.
(2) Signal capture devices: the location information provided by the sensor needs to be captured by the signal capture device. Usually, it is a Dina Luban or an infrared camera that captures electrical signals.
(3) Data transmission to processing equipment: all the captured signals are sent to the computer for processing and correction, thus forming a signal language that can be read by the computer, and bound with the model in the computer to complete the entire process of computer capture.
(4) Performer: A person who assists the system in motion capture by wearing a device to perform a series of prescribed actions.
2.2. Different Approaches towards motion capture technique
With the development of technology, people have already realized motion capture technology in various ways. According to the technical principle, it can be roughly divided into mechanical type, acoustic type, electromagnetic type, inertial sensor type, and optical type. Among them, the optical type can be further subdivided into marked point type and non-marked point type. Considering the cost and practical effect, optical motion capture and inertial sensor motion capture are the most widely used [2].
2.2.1. Mechanical motion capture. Mechanical motion capture systems rely on mechanical devices to record the movement of objects. A typical device will consist of multiple joints and links. The joints of the device are rotatable, the device can simulate the shape of different objects and the required posture according to the high demand, to be locked, and the built-in angle sensor will measure and record the angle change of the joint. The connecting rod records the change in length. These data will be processed by the computer to calculate the corresponding posture and motion, and then transmitted to the model in the computer to replicate the real action.
The implementation cost of the method is low, the collection of motion information is relatively accurate, real-time measurement can be performed according to motion, and multiple objects can be captured simultaneously. However, due to the need to wear the device at all times, when capturing human movements, the movement of the person wearing the device will be greatly restricted, resulting in impaired movement continuity. Therefore, this method has certain limitations in simulating real motion [3].
2.2.2. Acoustic motion capture. In acoustic motion capture, the signal transmitting device is an ultrasonic sound generator, and the receiving system is generally a plurality of ultrasonic probes. The travel time of the acoustic signal can be used to locate the distance the sound has traveled, and together multiple probes can calculate the position of the sound generator relative to the receiver. Just as we can roughly judge the direction of the sound by the sound we hear in our ears in real life, the ultrasonic probe can solve the spatial coordinates of the sound generator.
The biggest advantage of this method is its low cost. However, the shortcomings are also very obvious, because the sound wave is easily affected by the environment, and is easily affected by noise or multiple reflection factors, resulting in poor accuracy.
2.2.3. Electromagnetic motion capture. This method requires a magnetic field emission source. The emission source generates a regularly distributed electromagnetic field in space, and a moving object placed to receive the sensor will move in the magnetic field. As the object moves in the magnetic field, the sensor transmits the signal to the signal processing unit through a cable or wirelessly, and calculates the position of each sensor based on the distribution law of the electromagnetic field and the sensor signal. The cost of the scheme is reasonable, the real-time performance is good, and it is relatively simple to use.
However, the electromagnetic field itself is very sensitive to metal objects, and the electromagnetic field changes caused by metal objects will greatly affect the accuracy. Secondly, it is also difficult to capture fast movements and complex movements. Therefore, the application of electromagnetic motion capture also has limitations [4].
Considering the limitations of use, the two most widely used motion capture technologies today are optical motion capture and inertial motion capture.
2.2.4. Optical motion capture. Optical motion capture is mainly based on the principle of computer vision, and information capture is completed by tracking a certain feature point of the target through cameras from different angles. Just imagine the motion of any point in space, and if two cameras track it simultaneously from different angles, you can completely determine how that point moves in space. Because one camera can only capture the motion trajectory on one plane, the second camera can capture its stereoscopic motion, so we need at least two cameras to capture it. And when the two cameras shoot continuously at a high enough rate at the same time, we can completely track the 3D motion trajectory of the object.
In optical motion capture, only using the camera to detect the information of the feature points of the object is called markerless optical motion capture, and adding markers on the object as sensors are called optical motion capture.
2.2.5. Markerless Optical Capture. The principle of markerless optical motion capture is firstly based on the motion capture of video images. In this method, the computer detects the target object in the two-dimensional image, to extract the coordinates of the required joint points in the two-dimensional plane, and then combines multiple cameras to obtain the three-dimensional space from various measurements. information in. However, a given image may contain a lot of interfering information, which affects the computer's ability to extract the target, and is slow to render in real-time. Above this, it is even more difficult to locate the joint points that need to be captured. Therefore, to extract the target information more conveniently, we introduced an infrared camera so that the target image and the background can be separated by active heat source irradiation so that the target detection speed is greatly improved. However, because the irradiation direction of the heat source is fixed, the movement of the target is limited, and it is difficult to capture all-around motion. Similarly, in the absence of any sensor, we lack the means to clarify the reference information of joint points, so it is difficult to calculate and confirm the position of joint points only through images, and the resulting accuracy error is difficult to overcome.
2.2.6. Marker-Based Optical Capture. As mentioned above that markerless optical capture is difficult to extract the feature information of joint points so it is difficult to extract and locate directly in the computer. And markup can solve these problems very well. The fundamental method is to stick the optical identification points on the target joints so that multiple cameras can directly track the motion trajectory of the detection points and transmit information in real time, to achieve the purpose of motion capture technology. The advantages of this method are that the technology is relatively mature, the precision is high, the motion capture is also very accurate, the use is convenient and fast, and the cost is relatively low. However, its disadvantage is that the representation point may lose key information due to occlusion [5].
2.2.7. Inertial sensor-based motion capture. The inertial sensor-based motion capture system includes an attitude sensor, a signal receiver, and a data processing system. Among them, the attitude sensor is used to fix the main body parts of the human body, and the signal is transmitted to the data processing system for calculation using wireless transmission. The attitude sensor integrates measuring instruments of various physical quantities, including inertia, gravity, acceleration, magnetic induction, gyroscope, etc., which comprehensively records the motion information of this part. Combine the length and connection information of the sensor to restore the real movement mode.
The advantage of this method is that it is easy to operate, and the target performer is not restricted like mechanical motion capture and can perform outdoors. However, all recorded physical quantities are further calculated under the assumption of ground constraints, so positioning and calculation of motion cannot be performed after leaving the ground. In addition, the sensor cannot perform absolute positioning, in which the calculation of the physical quantity gives a relative physical motion trajectory, and the spatial positioning will also cause a lack of accuracy. In addition, the device itself, although significantly more portable than mechanical motion capture, still imposes a certain degree of restraint on the performer. And the cost will grow exponentially with the number of performers that need to be captured, which is also one of the drawbacks.
2.3. Emerging Technologies
With the maturity of different technologies, people are also looking for different ways to optimize motion capture technology, among which the motion capture technology combining different methods emerges as the times require. Among them, the light-inertial hybrid motion capture technology is an emerging motion capture technology that combines optical sensors and inertial sensors. Its advantage is to make up for the defects in various technologies to a certain extent so that more trial scenarios can be used. However, the solution is immature and the cost is unacceptable. The author believes that as people come up with more and more solutions to optimize motion capture technology [6].
3. Applications of Motion Capture Technology
3.1. Entertainment industry
In the entertainment industry, motion capture technology is widely used. From the movement of alien creatures in movies to NPCs in games to virtual anchors in virtual worlds, motion capture technology is used to make creatures that do not exist in the real world move like humans. The most classic example of this is Gollum in The Lord of the Rings. During the shooting, the plot related to Gollum will be repeated 3 times to achieve the most realistic effect. Similarly, the movie Teenage Mutant Ninja Turtles also uses motion capture technology to make the behavior of the turtles modeled by the computer more human-like [7].
3.2. Sports industry
Motion capture technology can track the performance of athletes in training and on the field so that professional coaching teams can provide better assistance to players. The motion capture technology can record every movement of the athlete, and play it frame by frame to help the coaching team analyze and guide their techniques, and can find out the hidden dangers of the athlete's physical condition for the change of the athlete's movement, to protect the athlete's body and provide the sports career [8].
3.3. Medical treatment
When the patient is undergoing rehabilitation training, the doctor usually observes the patient's movement posture with the naked eye. By the time doctors can detect the difference, the patient's movements are severely deformed. The introduction of motion capture technology can greatly improve the accuracy of the patient's motion posture detection, help doctors detect those small changes that are difficult to capture with the naked eye, and help doctors more timely and comprehensively understand the patient's condition and provide corresponding help before serious deformities occur [9].
3.4. Digital protection
Motion capture technology can digitally preserve artist performance records. Taking Chinese traditional cultural opera as an example, let the performers wear motion capture equipment to perform the opera, we can completely save all the performance data of the related opera, to protect it electronically. Using it as data, we can also make computer models perform similar actions, or let different people watch operas together through virtual reality [10].
4. Conclusion
Evolving from the idea to make the virtual world as vivid as possible, we have already developed multiple methods to approach our demand. This article sums up all the mainstream motion capture methodologies with their features for readers to examine the fundamental process so far. From the examination, the most useful approaches are optical motion capture and inertia sensor-based motion capture. Due to different constraints including cost, accuracy, usage scenario, acoustic motion capture, mechanical motion capture, and electromagnetic motion capture are being excluded for wider use. The main limitation of optical motion capture is the occlusion of the marker in marker-based and the problem to trace the target in markerless motion capture.
In future studies, an interesting study field may be how to extract target information without using markers by optimizing the way a computer recognizes a picture. Another way to optimize the methods is to try to find new emerging skills like the optical inertial hybrid mentioned in the article [6]. By improving the speed of processing images and the sensitivity of motion trackers, motion capture can be improved thus creating more vivid computer animation. Besides, motion capture techniques can be used as the foundation of facial expression capture techniques. Similar to motion capture, facial expressions requires tracing the target on people's face. The difference is that facial expressions can be subtle enough that we need more sophisticated capture techniques. Improving accuracy can help people to create vivid facial expressions on computer models.
The limitation of the article is that it provides a brief introduction to different methodologies without much theory explanation. Each method is highly-math-based and requires complex algorithms to process data. The article failed in explaining in-depth science-based explanation but focused on an outline of the existing technologies so that readers can have an overview of the field without much scientific background.
References
[1]. Guangtian Shi, Yongsheng Wang & Shuai Li: Human Motion Capture System and its Sensor Analysis. Sensors and Transducers, pp.172, 206-212 (2014).
[2]. Kinerd, NicholasKinerd: Motion Capture Study of Human Movement Recognition - Clemson University. https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=2512&context=all_theses, (2012).
[3]. Rauhl, M. Review on Motion Capture Technology - Global Journals. https://globaljournals.org/GJCST_Volume18/4-Review-on-Motion-Capture-Technology.pdf (2018).
[4]. Gutemburg, Guerra-Filho1.“Optical Motion Capture: Theory and Implementation.” CiteSeerX, https://citeseerx.ist.psu.edu/viewdoc/citations?doi=10.1.1.106.7248.
[5]. Gregor, M. & Horejsi, Petr & Michal, Šimon. Case study: Motion capture for ergonomics, pp.468-476 (2015).
[6]. Xi Chen, et al.: Research on Motion Capture Technology of Unmarked Optical Measurement and Inertial Measurement Fusion-- “Guangdong Science and Technology”, https://www.cnki.com.cn/Article/CJFDTotal-GDKJ202101042.htm (2021).
[7]. Hongxin Hu, Ming Cui: Development Scheme of Mobile Campus Information Integration Platform Based on Android. Development Scheme of Mobile Campus Information Integration Platform Based on Android | Atlantis Press, Atlantis Press, https://www.atlantis-press.com/proceedings/isccca-13/5870 (2013).
[8]. Pueo, Basilio & Jimenez-Olmedo, Jose. Application of motion capture technology for sport performance analysis. Retos: nuevas tendencias en educación física, deporte y recreación. pp.241-247 (2017).
[9]. Jun Zhang, Bo Zhu. Medical Rehabilitation Detection System Based on Motion Capture-- Electronic World, http://www.cnki.com.cn/Article/CJFDTotal-ELEW201808101.htm (2018).
[10]. Hongwei, Wang, et al. Depicting the digital protection of Liyuan Opera virtual reality based on motion capture technology, Journal of Quanzhou Normal University, (2021).
Cite this article
Xin,Z. (2023). The existing motion capture technologies and their application. Applied and Computational Engineering,13,7-12.
Data availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
Disclaimer/Publisher's Note
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
About volume
Volume title: Proceedings of the 5th International Conference on Computing and Data Science
© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and
conditions of the Creative Commons Attribution (CC BY) license. Authors who
publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons
Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this
series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published
version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial
publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and
during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See
Open access policy for details).
References
[1]. Guangtian Shi, Yongsheng Wang & Shuai Li: Human Motion Capture System and its Sensor Analysis. Sensors and Transducers, pp.172, 206-212 (2014).
[2]. Kinerd, NicholasKinerd: Motion Capture Study of Human Movement Recognition - Clemson University. https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=2512&context=all_theses, (2012).
[3]. Rauhl, M. Review on Motion Capture Technology - Global Journals. https://globaljournals.org/GJCST_Volume18/4-Review-on-Motion-Capture-Technology.pdf (2018).
[4]. Gutemburg, Guerra-Filho1.“Optical Motion Capture: Theory and Implementation.” CiteSeerX, https://citeseerx.ist.psu.edu/viewdoc/citations?doi=10.1.1.106.7248.
[5]. Gregor, M. & Horejsi, Petr & Michal, Šimon. Case study: Motion capture for ergonomics, pp.468-476 (2015).
[6]. Xi Chen, et al.: Research on Motion Capture Technology of Unmarked Optical Measurement and Inertial Measurement Fusion-- “Guangdong Science and Technology”, https://www.cnki.com.cn/Article/CJFDTotal-GDKJ202101042.htm (2021).
[7]. Hongxin Hu, Ming Cui: Development Scheme of Mobile Campus Information Integration Platform Based on Android. Development Scheme of Mobile Campus Information Integration Platform Based on Android | Atlantis Press, Atlantis Press, https://www.atlantis-press.com/proceedings/isccca-13/5870 (2013).
[8]. Pueo, Basilio & Jimenez-Olmedo, Jose. Application of motion capture technology for sport performance analysis. Retos: nuevas tendencias en educación física, deporte y recreación. pp.241-247 (2017).
[9]. Jun Zhang, Bo Zhu. Medical Rehabilitation Detection System Based on Motion Capture-- Electronic World, http://www.cnki.com.cn/Article/CJFDTotal-ELEW201808101.htm (2018).
[10]. Hongwei, Wang, et al. Depicting the digital protection of Liyuan Opera virtual reality based on motion capture technology, Journal of Quanzhou Normal University, (2021).