A review of motion generation technology

Research Article
Open access

A review of motion generation technology

Zhe Yang 1*
  • 1 Shanghai DianJi University    
  • *corresponding author yz91570@163.com
Published on 31 January 2024 | https://doi.org/10.54254/2755-2721/30/20230073
ACE Vol.30
ISSN (Print): 2755-273X
ISSN (Online): 2755-2721
ISBN (Print): 978-1-83558-285-5
ISBN (Online): 978-1-83558-286-2

Abstract

Nowadays, deep learning and neural network-related research play a very important role in the widely use of artificial intelligence -related technologies, Among them, the hot development in the direction of generative adversarial networks (GAN) has given birth to many generation-related techniques. For example, MoCoGAN is based on the implementation principle of GAN, which enables video generation of different actions of the same character or the same action of different characters, through the method that decompose video into actions and content. This paper introduces the history and principles of MoCoGAN, starting from the prospect of using MoCoGAN in artificial intelligence (AI) industry and the technical challenges that need to be overcome in the future application of action generation. Besides, this paper also discusses the two main issues of how to improve the quality of video generation using MoCoGAN and the input conditions that are the most central problem to GAN networks. By summarizing the optimization solutions of other researchers in these two areas in recent years, this paper searches the core problems need to be solved and propose a broad prospect for future video generation techniques that can be implemented by using MoCoGAN in human-computer interaction (HCI) area.

Keywords:

Artificial Intelligence, Deep Learning, MoCoGAN, Generate Motion, HCI

Yang,Z. (2024). A review of motion generation technology. Applied and Computational Engineering,30,68-73.
Export citation

References

[1]. Goodfellow I., Pouget-Abadie J., Mirza M., et al. Generative adversarial networks [J]. Communications of the ACM, 2020, 63(11): 139-144.

[2]. Chan C., Ginosar S., Zhou T., et al. Everybody dance now [C]// Proceedings of the IEEE/CVF international conference on computer vision. 2019: 5933-5942.

[3]. Vondrick C., Pirsiavash H., Torralba A. Generating videos with scene dynamics [J]. Advances in neural information processing systems, 2016, 29.

[4]. Zhou Y., Berg T. L. Learning temporal transformations from time-lapse videos [C]// Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14. Springer International Publishing, 2016: 262-277.

[5]. Saito M., Matsumoto E., Saito S. Temporal generative adversarial nets with singular value clipping [C]// Proceedings of the IEEE international conference on computer vision. 2017: 2830-2839.

[6]. Tulyakov S., Liu M. Y., Yang X., et al. Mocogan: Decomposing motion and content for video generation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 1526-1535.

[7]. Dandi Y., Das A., Singhal S., et al. Jointly trained image and video generation using residual vectors [C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2020: 3028-3042.

[8]. Unterthiner T., van Steenkiste S., Kurach K., et al. FVD: A new metric for video generation [J]. 2019.

[9]. Kimura S., Kawamoto K. Conditional mocogan for zero-shot video generation [J]. arXiv preprint arXiv:2109.05864, 2021.

[10]. Mirza M., Osindero S. Conditional generative adversarial nets [J]. arXiv preprint arXiv:1411.1784, 2014.

[11]. Wang Y., Bilinski P., Bremond F., et al. G3AN: Disentangling appearance and motion for video generation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 5264-5273.

[12]. Siarohin A., Lathuilière S., Tulyakov S., et al. Animating arbitrary objects via deep motion transfer [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 2377-2386.


Cite this article

Yang,Z. (2024). A review of motion generation technology. Applied and Computational Engineering,30,68-73.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 2023 International Conference on Machine Learning and Automation

ISBN:978-1-83558-285-5(Print) / 978-1-83558-286-2(Online)
Editor:Mustafa İSTANBULLU
Conference website: https://2023.confmla.org/
Conference date: 18 October 2023
Series: Applied and Computational Engineering
Volume number: Vol.30
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Goodfellow I., Pouget-Abadie J., Mirza M., et al. Generative adversarial networks [J]. Communications of the ACM, 2020, 63(11): 139-144.

[2]. Chan C., Ginosar S., Zhou T., et al. Everybody dance now [C]// Proceedings of the IEEE/CVF international conference on computer vision. 2019: 5933-5942.

[3]. Vondrick C., Pirsiavash H., Torralba A. Generating videos with scene dynamics [J]. Advances in neural information processing systems, 2016, 29.

[4]. Zhou Y., Berg T. L. Learning temporal transformations from time-lapse videos [C]// Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14. Springer International Publishing, 2016: 262-277.

[5]. Saito M., Matsumoto E., Saito S. Temporal generative adversarial nets with singular value clipping [C]// Proceedings of the IEEE international conference on computer vision. 2017: 2830-2839.

[6]. Tulyakov S., Liu M. Y., Yang X., et al. Mocogan: Decomposing motion and content for video generation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 1526-1535.

[7]. Dandi Y., Das A., Singhal S., et al. Jointly trained image and video generation using residual vectors [C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2020: 3028-3042.

[8]. Unterthiner T., van Steenkiste S., Kurach K., et al. FVD: A new metric for video generation [J]. 2019.

[9]. Kimura S., Kawamoto K. Conditional mocogan for zero-shot video generation [J]. arXiv preprint arXiv:2109.05864, 2021.

[10]. Mirza M., Osindero S. Conditional generative adversarial nets [J]. arXiv preprint arXiv:1411.1784, 2014.

[11]. Wang Y., Bilinski P., Bremond F., et al. G3AN: Disentangling appearance and motion for video generation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 5264-5273.

[12]. Siarohin A., Lathuilière S., Tulyakov S., et al. Animating arbitrary objects via deep motion transfer [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 2377-2386.