Overview of capability requirement analysis methods for operational concept development

Research Article
Open access

Overview of capability requirement analysis methods for operational concept development

Jing An 1* , Xue chao Zhang 2 , Chun lan You 3
  • 1 Joint Logistics College of NDU, Beijing, China, 100000    
  • 2 Joint Logistics College of NDU, Beijing, China, 100000    
  • 3 Joint Logistics College of NDU, Beijing, China, 100000    
  • *corresponding author anj21_2000@sina.com
ACE Vol.5
ISSN (Print): 2755-2721
ISSN (Online): 2755-273X
ISBN (Print): 978-1-915371-57-7
ISBN (Online): 978-1-915371-58-4

Abstract

Scientific and reasonable analysis and determination of operational capability requirements can not only optimize and improve the operational concept, but also ensure the transformation and application of the operational concept. The development and construction of the traction force and the improvement of operational capability play a key role in the transformation of operational theory to actual combat capability. It is urgent to study scientific and applicable operational capability requirements analysis methods to support the development process of the operational concept. On the basis of defining the components of operational capability requirements, this paper combs, summarizes, analyzes and compares the main operational capability requirements analysis methods, points out the problems of existing analysis methods in combination with current research, and summarizes and prospects the next research direction.

Keywords:

operational concept development, Capability requirement analysis method, overview

An,J.;Zhang,X.C.;You,C.L. (2023). Overview of capability requirement analysis methods for operational concept development. Applied and Computational Engineering,5,54-61.
Export citation

1. Introduction

The operational concept is to study the essential laws of operational elements, such as operational conditions and adversaries, and abstract and summarize the proposed solutions to operational problems by judging the established operational concept and intention under the specific space-time conditions in the future. In the development process of operational concept, it is necessary to define three types of elements. The first type is the description of operational problems, including the operational background, operational purpose, operational tasks, operational environment, combat opponents, time setting, conflict scenarios, etc. of the operational concept under specific space-time conditions in the future. The second is the solutions to specific operational problems, including the guiding ideology of operational concepts, operational principles, operational forces, operational activities, and operational concepts. The third is the operational capability requirements, that is, the key capabilities and their indicator requirements required for conversion of landing solutions to achieve operational effects, as well as the current capability gap analysis.

The capability requirement analysis oriented to the development of operational concepts is the process of analyzing solutions to specific operational problems, and studying and clarifying the requirements of various operational capabilities. As the core link of the development of operational concept, the analysis of operational capability requirements needs to go through the process from strategic analysis to detailed design, from subjective abstraction to objective description, from general principles to specific guidance, and from theoretical research to transformation and application. It is not a simple logical thinking process, but a pioneering complex scientific system project. It needs to adopt the engineering, systematic and modern development concept, rely on simulation Advanced technical means, such as war games, shall be used to study scientific and applicable analysis methods for operational capability requirements to ensure that the analysis is practical, comprehensive, scientific and reasonable. In view of the important role of operational capability requirement analysis methods in the development of operational concepts, and the current development stage of this field, this paper analyzes and compares them, proposes existing problems, and looks forward to the research direction. The main contributions of this review are as follows: First, propose the definition of operational concept, and define the elements of operational concept and operational capability requirements; The second is to analyze the advantages and disadvantages of common methods, and analyze the applicability of each method in combination with the particularity of operational capability requirement analysis, and point out the existing shortcomings; Third, in view of the shortcomings of the existing methods, the development direction of the methods is prospected to provide theoretical reference for proposing a scientific and applicable capability requirement analysis method oriented to the development of operational concepts.

2. Research status

At present, there are various of method to analysis operational requirements. Such as analytical method, multi-view analysis method, simulation experiment analysis method, machine learning analysis method, etc.

2.1. Analytic method

Analytic method, also known as analytical method, is a solution method based on strict axiom and logic system and mathematical theory. The analytic method is applied to combat capability demand analysis. Based on the assumptions and simplified influencing factors, the demand analysis problem is converted into a strict mathematical or computer model. Through the solution, the absolute and mathematical correct solution or optimal solution is obtained, so as to draw the analysis conclusion of capability requirements. The outstanding characteristics of this method are strict form, standard process and strong operability [1]. However, due to few factors and strict conditions, the analysis results are often abstract, random and poor accuracy. For example, Rand Corporation used a simplified mathematical model to roughly evaluate and analyze the demand for the mainland's force projection capability, the key issue in the landing operation, in "a balance problem - the political background and military profile of the conflict ", and concluded that the mainland's sea projection capability was insufficient and the landing operation was at risk. In the modeling process, the restrictions are too strict, and the analysis conditions are too simplified and rough, for example, the conditions such as naval coordination are not considered, which greatly affects the analysis effect. Qi Xiao-gang et al. put forward a measurement model for the emergence of combat capability in the paper [2], and analyzed the demand for combat capability by taking an aircraft carrier formation's anti-ship operation as an example. However, because the research object is selected randomly, this method is not universal.

2.2. Multi-view

As a common analysis method for complex systems, multi-view analysis is applied to the analysis of operational capability requirements. The core idea is to decompose the operational concept to be analyzed into several relatively independent views from different perspectives. Each view reflects the operational capability requirements of the operational concept from one dimension, and then all views are summarized to form the analysis of the overall capability requirements of the operational concept. At present, the representative multi view analysis methods are DoDAF2.0 in the United States, MoDAF in the United Kingdom and NAF in NATO. For example, the U.S. Department of Defense, based on the architecture description of the joint operations concept, uses the Joint Capabilities Integration and Development System (JCIDS) to analyze the gaps in operational capabilities and their priorities, and guide the determination of the integrated solution of the DOTMLPF-P framework. Literature [3] [4] [5] [6], based on DoDAF2.0 model, respectively, proposed the capacity requirement analysis methods of structural design and semi quantitative analysis, and analyzedThe operational capability requirements of such operational concepts as multi area warfare and ground unmanned warfare are also presented. Literature [7] decomposes combat activities based on multi view analysis method, and then realizes the mapping from mission tasks to capability requirements through meta activities. However, it only focuses on the capability requirements of the equipment system of systems, and does not describe the combat capability requirements of the system of systems. Literature [8] has constructed a relatively complete combat capability requirement of mission system, capability system and equipment system, but has not given the mapping relationship between each system. Document [9] proposed a modeling method for combat capability requirements based on the commander's perspective, which is easy to understand, but has strong subjectivity.

2.3. Simulation experiments

The analysis process of simulation experiment method mainly includes: preparation, implementation of simulation experiment and analysis of experimental results. This method is applied to the capability requirement analysis of the combat concept. The core idea is: first, set the experimental conditions and establish the simulation model for the combat scenario space; Secondly, the simulation experiment is carried out by adjusting the control experiment variables, and the experimental results under different operational constraints are collected; Finally, through the statistical analysis of the experimental process and results, the results of capacity demand analysis are obtained. The simulation experiment method can help to understand, refine and verify the causal effects between different combat capability generation factors, and explore potential expected and unexpected results. For example, in the development process of the operational concept of "Joint Operation Intervention", the United States, land, sea, air and other services formed the Final Report of Limited Target Experiment of Joint Operation Intervention Concept through experimental analysis, and determined 36 capability requirements; In 2021, US think tank strategyThrough many deduction experiments with the budget evaluation center, we tested the force formation, command and control, decision-making action, etc. of man-machine integration, and explored the capability requirements of mosaic warfare [10] [11]. Professor Si Guangya of China once put forward the view that "war experiments can help us understand the rules of warfare and new concepts of warfare" in the literature [12]. His team first proposed the method framework of "exploratory simulation experiment analysis based on data farming" in the literature [13] for the analysis of operational capability requirements.

2.4. Machine learning

Machine learning is an operational capability requirement analysis method based on theoretical analysis, statistical analysis, and experimental analysis, which adds another effective means. The core idea of this analysis method for operational capability requirement analysis is to use machine learning to analyze and mine large sample and high-dimensional operational data. The general process is: firstly, conduct qualitative analysis on the operational concept, define the data requirements and determine the acquisition mode of multi-source and multi-mode data based on the construction of analysis and evaluation criteria; Second, the large-scale data set required for the collection capability analysis shall be organized according to different standards or methods such as purpose, time and space; Finally, machine learning algorithms such as deep learning and reinforcement learning are used to analyze the mining data, explore and clarify the development trend and changes, and determine the various elements of the capacity demand. The advantage of the data analysis method is that it can better build the evaluation index system, explore the rules of the operational mechanism, and predict the changes of the battlefield trend. For example, the 2018It was mentioned in the Summary of Artificial Intelligence Strategy of the Ministry of National Defense in that artificial intelligence technologies such as machine learning should be applied to many fields such as intelligence reconnaissance, logistics support, command and control, unmanned autonomous operations, and operational capability analysis and evaluation [14]. In 2021, the RAND report [15] recorded the "Exploring Social Media and Operational Data Streams" project sponsored by the U.S. Army Special Operations Command, which developed a standardized method for operational analysis of special operations forces, and described how to use machine learning to support the analysis of war operations. Chinese military researchers mainly apply machine learning to knowledge extraction, model construction, index mining, simulation and deduction, and visual analysis of operational capability requirement analysisAnd knowledge representation [16]. Literature [17] proposed to build an intelligent evaluation model of decision-making effect based on stack self coding network (SAE) to achieve intelligent evaluation of decision-making effect. Literature [18] proposed a backtracking analysis method of defense and control system effectiveness based on forced sparse self coding neural network to achieve in-depth analysis of the emerging mechanism of air defense operation system capability.

3. Comparison and analysis of capability requirements analysis methods for operational concept development

As the "Capability Requirements Analysis for Operational Concept Development" has certain particularity in terms of analysis objectives, analysis processes and conditions compared with the analysis of operational plans, operations and other capabilities, we have analyzed the characteristics and applicability of the above methods and proposed the shortcomings of existing methods.

Compared with the objectives, conditions, process characteristics and method requirements of the operational concept capability requirements analysis, the above methods also have the following shortcomings to provide targeted method support for the operational concept development oriented capability requirements analysis:

First, the analysis process is unidirectional, unable to meet the analysis needs of reverse exploration. Through the above analysis, the basic principle of the existing analysis and evaluation methods can be expressed as \( A=Fm(\vec{W}) \) : A is the output conclusion, \( \vec{W} \) is the input condition, and \( Fm \) is the analysis process of various analysis methods. Obviously, \( A=Fm(\vec{W}) \) is not reverse and \( Fm \) often has black box unexplainability. Therefore, it cannot support the demand for input \( \vec{W} \) optimization based on \( A \) . For example, literature [2] analyzed the anti-ship capability requirements of aircraft carrier formation through emergent measurement model. This method focuses on statistical data analysis and lacks two-way capability analysis related to mission effectiveness.

The second is the certainty of the input conditions of the analysis method, which cannot meet the analysis requirements of quickly adapting to high-dimensional feature changes. At present, the analytical model, index system, simulation and deduction methods are used to analyze the number, type and boundary of input conditions. Using it as input for multidimensional feature mining and analysis will face problems such as input space explosion, large amount of calculation, large amount of data, and complex analysis process, which cannot meet the needs of rapid adaptation. The first is the contradiction between the exploration space and the limited computing capacity. The research on the capability requirement analysis for the development of operational concepts is complex. It is necessary to comprehensively consider the equipment and non equipment factors in the operational system. Too many experimental factors and discrete and uncontrollable experimental levels lead to a sharp increase in the exploration space and conflict with the limited computing capacity. The second is the contradiction between excessive space compression and analysis accuracy. To ensure the exploration rules of operational capability requirement analysis Modulus, the sample space of the analysis problem must be compressed and simplified, which may lead to the distortion of demand analysis.

Third, the application conditions of the analysis method are incomplete, which cannot meet the analysis requirements of the real threat environment and operational conditions. In the practical application of existing methods, if we want to meet the antagonism analysis requirements of the real threat environment and combat conditions, either we need to establish relevant analytical models based on actual combat data and empirical data, or we need effective model algorithms and experimental systems to support simulation experiment analysis. However, due to the lack of operational data, or the difficulty in constructing the basic environment of simulation experiments such as battlefield environment data, simulation model library, and simulation system, the application of methods is limited, and the controllability of analysis methods and the credibility of analysis conclusions are reduced. First of all, the new combat concept is oriented to the future. For new equipment, new forces, and new methods of warfare, new simulation models are often required. The demand for model construction is large. However, without the support of empirical data, model construction is difficult and takes a long time; Secondly, in order to give consideration to the overall combat concept andFor local capability analysis, MRM (Multi resolution model) needs to be established to ensure the accuracy of the evaluation results and improve the analysis efficiency, but there is no effective multi-resolution modeling method at present; Finally, there is a contradiction between the high accuracy requirement of the model and the lack of verification methods of the model. The model is the data source for operational conceptual capability analysis. If not verified, the accuracy is not high, which will directly lead to the analysis conclusion being questioned. For example, the document [19] proposed the method of capability requirement analysis based on C4ISR framework and oriented to mission architecture. However, due to the need to build a large number of models and the lack of support from actual combat data in the modeling process, the modeling difficulty is increased to a certain extent and the feasibility of the method is reduced.

4. Research prospect

In view of the above analysis, the next step is to innovate a scientific and applicable operational capability requirement analysis method that can be replicated and popularized based on reductionism, holism, qualitative analysis and quantitative calculation in complex system science and oriented to the process of operational concept development, so as to improve the scientific and systematic development of operational concepts. The main research directions include:

4.1. Research on model construction and formal rescription method of operational concept

The capability requirement analysis for the development of operational concept is a process of cycle iteration, from coarse to fine, and deepening. It is often difficult to effectively clarify the specific content and organizational logic of the complex combat capability requirements, as well as the interaction between the combat capability requirements and other elements of the combat concept, and it is also impossible to achieve the iterative update of the results of the combat capability requirements analysis by simply relying on the macro abstract text description. Therefore, it is urgent to use scientific and systematic engineering methods, give consideration to micro actions and macro effects, and carry out research on modeling and description methods of operational concepts. Through systematic structural modeling and formal description of operational concepts with low structure, strong abstraction, and high complexity, we can better analyze operational problems, clarify solutions, and clarify the composition, content, and logic of operational capability requirements, Provide support for the analysis of operational capability requirements. Focus on the following three aspects:

First, build a multi perspective architecture model. The elements of the operational concept are diverse and complex, which can usually be regarded as a system with indestructibility, uncertainty, emergence, etc. In order to better describe this system and its elements and support the development and analysis of operational concepts, it is necessary to model the architecture of operational concepts from different perspectives.

The second is to adopt the modeling method focusing on capability requirements. The operational concept is a visual expression of "future operations". During the modeling process, it is necessary to face the future, take the operational problems as the logical starting point, take the demand analysis of future operational capabilities as the logical main line, and always focus on the ability of combat forces to complete specific operational tasks.

The third is to rely on a unified standardized model framework. Operational concept development is a pioneering complex scientific system engineering that requires long-term cooperation. It should adopt an engineering and systematic development concept and rely on a standardized model framework for modeling to ensure the effectiveness, continuity and compatibility of the model in the whole life cycle of operational concept development.

4.2. Research on methods of operational data in the confrontation environment

The basis for analyzing the operational capability requirements in a qualitative and quantitative way is the operational data, especially in the confrontation environment. This is because the development of the operational concept is to solve operational problems, and must focus on certain operational scenarios and confrontation environments. However, as far as the current situation is concerned, there is little practical experience in the development of operational concepts, few relevant confrontation exercises, and a serious lack of empirical data in real threat scenarios. Therefore, it is urgent to focus on the application of operational experiment theories and methods in the development of operational concepts, build confrontation scenarios, design experimental scenarios, improve support tools, implement operational experiments, generate and collect antagonistic data samples for analysis, and lay the foundation for data mining, machine learning and other analysis methods by relying on multiple methods and means such as war games, simulation experiments, and live military exercises. Focus on the following two aspects:

First, design of experiments (DOE). The analysis of operational capability requirements relies on the in-depth exploration and research of operational issues. Therefore, it is necessary to develop simulation experiment scenarios based on the system model of operational concept and around the needs of quantitative analysis, build or select simulation models and simulation systems, use exploratory analysis strategies, optimize the design and control of computer simulation experiments, and then support the acquisition of massive experimental data, The key operational capability requirements related to the overall objectives of the war are mined from the data results.

The second is the research of data preprocessing and data enhancement methods. Data is the basis of analysis. According to the index system of capability analysis, relying on various methods and means such as war games, simulation experiments, and live military exercises, we can construct confrontation scenarios, design experimental scenarios, improve support tools, conduct operational experiments, generate and collect antagonistic data samples for analysis, and analyze various raw data collected in the simulation experiment stage, And through machine learning technologies such as GAN, we expand and enhance the simulation data, establish the mapping relationship between the indicator system and the simulation data, supplemented by data generation, data fusion and other methods, to form a large-scale training data set for analysis and model training, laying the foundation for data mining, machine learning and other analysis methods.

4.3. Research on capacity analysis method combining qualitative and quantitative analysis

The objective of the capability requirements analysis for the development of operational concepts is to comprehensively use the analytical, experimental, qualitative, quantitative and other multi category problem solving methods and tools to scientifically and efficiently gradually refine and quantify the formal description of the capability requirements in the operational concept architecture, including capability indicators, indicator thresholds, and relationships between indicators. Therefore, it is urgent to study the methods and means that can support the capacity demand from qualitative analysis to quantitative approximation, from quantitative calculation to qualitative analysis, and gradually focus and refine. Focus on two aspects:

First, from qualitative analysis to quantitative approximation. Data analysis needs to use people's experience and insight to put forward assumptions. Therefore, in the process of operational capability demand analysis, large sample training data is obtained from qualitative analysis of the operational concept as a starting point, based on commander's experience, on the basis of studying qualitative relations, using forward analysis methods such as analytic hierarchy process and simulation deduction, from qualitative to quantitative approximation, to support subsequent analysis.

The second is from quantitative calculation to qualitative analysis. Based on the training data set, machine learning and other data analysis methods are used to build a machine learning capability agent model. Through training, accurate capability agent model is fitted to replace the simulation experiment. Interactive learning is conducted between the trained capability agent model and the deep reinforcement learning framework to continuously reduce the uncertainty of operational problems until a satisfactory solution of capability requirements that meets or approaches the requirements of operational purposes is found, and qualitative analysis and interpretation are conducted to realize the forward and reverse output analysis of operational capability requirements and capability generation mechanism.

5. Conclusion

The scientific and applicable analysis method of operational capability requirements can optimize and improve the operational concept, ensure the transformation and application of the operational concept, and form a strong support for the development process of the operational concept. This paper takes the combat capability requirement elements as the starting point, combs and summarizes the research status of the combat capability requirement analysis methods, combines the particularity of the combat capability requirement analysis, compares and analyzes the advantages, disadvantages and applicability of each method, looks forward to the development direction of the methods, and provides theoretical reference for proposing a scientific and applicable capability requirement analysis method oriented to the development of combat concepts.


References

[1]. Chen Shitao, Sun Peng, Li Daxi. Analysis of the New Concept of Operations [M]. Xi'an: Xi'an University of Electronic Science and Technology Press, October, 2019

[2]. Qi Xiaogang, Liu Xuexing. Emergence measurement model of weapon equipment system based on structural equation model [J]. Journal of Military Industry, 2020, 41 (2): 406-416

[3]. MAJ Lindsay S. Maples .Sustainment Considerations for the Multi-Domain Battle[D]:[Master’s Thesis].Fort Leaveworth,KS:U.S.Army Command and General Staff College.2018:1-3

[4]. Zhang Ziwei, Li Liang, Dong Zhiming, Wang Yifei, Duan Li. Research on the construction method of combat effectiveness simulation evaluation index of combat concept traction [J]. Journal of System Simulation. 2021

[5]. Du Guohong. Characteristics of the Development of US Military Operational Concept [J]. National Defense Science and Technology, 2020 (4)

[6]. Ding Wei, Geng Li. Research on Optimization of Complex System Design Method Based on System of Systems Operational Requirements [C]. Proceedings of Complex System of Systems Engineering, 2019

[7]. Chen Wuying, Dou Yajie, Cheng Ben, et al. Research on generation of capability requirements of weapon equipment system based on operational activity decomposition [J]. System Engineering Theory and Practice, 2011, 31 (1): 154-163

[8]. Yu Tonggang, Sun Zhiming, Zhang Xiaokang, et al. Research on the generation process of equipment system requirements based on joint operation capability [J]. Journal of the Academy of Ordnance Engineering, 2009, 21 (3): 10-13

[9]. Xu Xiufeng, Si Guangya, Wang Yanzheng. A conceptual model framework for military operations based on OPM [J]. Command Control and Simulation, 2020, 42 (6): 1673-3819

[10]. United States Army Redstone Technical Test Center(RTTC). "What We Do; Modeling, Simulation, and Hardware/Human-in-the-Loop Technology Integrated into Testing."([cited15August2004])Available on the World Wide Web @ http://www.rttc.army.mil/whatwedo/primary_ser/modeling.htm

[11]. Doshi-Velez,Finale,and BeenKim. "Towards arigorous science of interpretable machine learning." arXivpreprintarXiv:1702.08608(2017).

[12]. Si Guangya, Yang Jingyu, Wang Yanzheng, Hu Xiaofeng Understanding and Thinking on the Concept of War Experiment [J] Military Operations Research and Systems Engineering, 2008, 22 (3): 14-19

[13]. Hu Runtao, Hu Xiaofeng Design of exploratory simulation analysis framework based on data cultivation [J] Computer simulation 2009 (01): 521-526

[14]. Joint Interoperability Test Command. Joint combat identification evaluation team,single integrated airpicture system engineer. Millennium challenge 2002 data management and analysis plan[R/OL],2002-07-27.[2013-08-20]

[15]. DANIEL EGEL, RYAN ANDREW BROWN, LINDA ROBINSON, MARY KATE ADGIE,JASMIN LÉVEILLÉ, LUKE J. MATTHEWS. Leveraging Machine Learning for Operation Assessment[R]. RAND Corporation, Santa Monica, Calif. 2021.

[16]. Ou Wei. An Intelligent Evaluation Model for the Decision Effect of Wargame Entities Based on Deep Learning [J]. Military Operations Research and System Engineering, 2018, 32 (04): 29-34

[17]. Guo Shengming, He Xiaoyuan, Wu Lin, et al. Effectiveness retrospective analysis method of air defense combat system based on forced sparse self coding neural network [J] Chinese Science: Information Science, 2015, 48:824-840, doi: 10.1360/N112017-00303

[18]. Li Xiaoxi, Chen Haoguang, Li Daxi, et al. Research on the combat effectiveness prediction model based on Elman neural network [J]. System Simulation. 2015.27 (1): 43-49

[19]. C4ISR Architecture Working Group. C4ISR Architecture Framework Version2.0[R]. U.S.: Department of Defense, 1997.


Cite this article

An,J.;Zhang,X.C.;You,C.L. (2023). Overview of capability requirement analysis methods for operational concept development. Applied and Computational Engineering,5,54-61.

Data availability

The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.

Disclaimer/Publisher's Note

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of EWA Publishing and/or the editor(s). EWA Publishing and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

About volume

Volume title: Proceedings of the 3rd International Conference on Signal Processing and Machine Learning

ISBN:978-1-915371-57-7(Print) / 978-1-915371-58-4(Online)
Editor:Omer Burak Istanbullu
Conference website: http://www.confspml.org
Conference date: 25 February 2023
Series: Applied and Computational Engineering
Volume number: Vol.5
ISSN:2755-2721(Print) / 2755-273X(Online)

© 2024 by the author(s). Licensee EWA Publishing, Oxford, UK. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. Authors who publish this series agree to the following terms:
1. Authors retain copyright and grant the series right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this series.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the series's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this series.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open access policy for details).

References

[1]. Chen Shitao, Sun Peng, Li Daxi. Analysis of the New Concept of Operations [M]. Xi'an: Xi'an University of Electronic Science and Technology Press, October, 2019

[2]. Qi Xiaogang, Liu Xuexing. Emergence measurement model of weapon equipment system based on structural equation model [J]. Journal of Military Industry, 2020, 41 (2): 406-416

[3]. MAJ Lindsay S. Maples .Sustainment Considerations for the Multi-Domain Battle[D]:[Master’s Thesis].Fort Leaveworth,KS:U.S.Army Command and General Staff College.2018:1-3

[4]. Zhang Ziwei, Li Liang, Dong Zhiming, Wang Yifei, Duan Li. Research on the construction method of combat effectiveness simulation evaluation index of combat concept traction [J]. Journal of System Simulation. 2021

[5]. Du Guohong. Characteristics of the Development of US Military Operational Concept [J]. National Defense Science and Technology, 2020 (4)

[6]. Ding Wei, Geng Li. Research on Optimization of Complex System Design Method Based on System of Systems Operational Requirements [C]. Proceedings of Complex System of Systems Engineering, 2019

[7]. Chen Wuying, Dou Yajie, Cheng Ben, et al. Research on generation of capability requirements of weapon equipment system based on operational activity decomposition [J]. System Engineering Theory and Practice, 2011, 31 (1): 154-163

[8]. Yu Tonggang, Sun Zhiming, Zhang Xiaokang, et al. Research on the generation process of equipment system requirements based on joint operation capability [J]. Journal of the Academy of Ordnance Engineering, 2009, 21 (3): 10-13

[9]. Xu Xiufeng, Si Guangya, Wang Yanzheng. A conceptual model framework for military operations based on OPM [J]. Command Control and Simulation, 2020, 42 (6): 1673-3819

[10]. United States Army Redstone Technical Test Center(RTTC). "What We Do; Modeling, Simulation, and Hardware/Human-in-the-Loop Technology Integrated into Testing."([cited15August2004])Available on the World Wide Web @ http://www.rttc.army.mil/whatwedo/primary_ser/modeling.htm

[11]. Doshi-Velez,Finale,and BeenKim. "Towards arigorous science of interpretable machine learning." arXivpreprintarXiv:1702.08608(2017).

[12]. Si Guangya, Yang Jingyu, Wang Yanzheng, Hu Xiaofeng Understanding and Thinking on the Concept of War Experiment [J] Military Operations Research and Systems Engineering, 2008, 22 (3): 14-19

[13]. Hu Runtao, Hu Xiaofeng Design of exploratory simulation analysis framework based on data cultivation [J] Computer simulation 2009 (01): 521-526

[14]. Joint Interoperability Test Command. Joint combat identification evaluation team,single integrated airpicture system engineer. Millennium challenge 2002 data management and analysis plan[R/OL],2002-07-27.[2013-08-20]

[15]. DANIEL EGEL, RYAN ANDREW BROWN, LINDA ROBINSON, MARY KATE ADGIE,JASMIN LÉVEILLÉ, LUKE J. MATTHEWS. Leveraging Machine Learning for Operation Assessment[R]. RAND Corporation, Santa Monica, Calif. 2021.

[16]. Ou Wei. An Intelligent Evaluation Model for the Decision Effect of Wargame Entities Based on Deep Learning [J]. Military Operations Research and System Engineering, 2018, 32 (04): 29-34

[17]. Guo Shengming, He Xiaoyuan, Wu Lin, et al. Effectiveness retrospective analysis method of air defense combat system based on forced sparse self coding neural network [J] Chinese Science: Information Science, 2015, 48:824-840, doi: 10.1360/N112017-00303

[18]. Li Xiaoxi, Chen Haoguang, Li Daxi, et al. Research on the combat effectiveness prediction model based on Elman neural network [J]. System Simulation. 2015.27 (1): 43-49

[19]. C4ISR Architecture Working Group. C4ISR Architecture Framework Version2.0[R]. U.S.: Department of Defense, 1997.